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1. IMTRODUCnON 

The problem of system identification has been studied intensively 
and has become one of the most active fields in engineering research. 

Some investigators have obtained theoretical results on parameter identifl- 
ability, convergence properties of parameter estimates [1,16,18,19], and 
regions of parameter identifiability [7]. There are also investigators 
working on developing computational algorithms for identifying certain 
control system parameters, for example, the stability and control deriva- 
tives of an aircraft from actual flight test data [17]. 

There are two main steps in solving cUe system identification 
problem: (1) Determine if the system parameters are identifiable, and 

(2) tf the parameters are identifiable, work out an algorithm for esti- 
mating the parameters. To answer (1), we have to establish the definition 
of identifiability first. The most significant recent work on parameter 
identifiability when there are stochastic disturbances present in the 
system are Tse [18] and Tse and Anton [19]. There are a number of 
algorithms that have been proposed for solving (2). 

Generally, if the system parameters are identifiable, they are only 
locally identifiable, i.e., we must have a sufficiently good initial 
estimate of the parameters such that the iterative estimation se<:;aence 
will converge to the true parameters. Herget [7] provided a procedure 
for computing explicit regions in the parameter space in which the Gauss- 
Newton method will converge to a unique solution. The systems considered 
by him were deterministic. 
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The definition of porameter Idont If lability when there are stochastic 
disturbances present has been given by Tse and Anton [19]. They said 
the paraneters are Identifiable If there exists a sequence of estimates 
Which Is consistent In probability. They also established the necessary 
and sufficient conditions for the unknown parameters to be Identifiable 
under some uniformity assumptions on the conditional density parametrized 
by the unknown parameters. Tse [18] also gave the definition of local 
parameter identifiabllity and proved that the positive definiteness of 
the average information matrix implies local parameter identifiabllity. 
The definition of local parameter Identifiabllity is that there exists 
an open region containing the vector of unknown parameters as an 
interior point and there exists a local estimation sequence in the 
closure of the region which is consistent in probability. Staley and 
Yue [16] established a similar concept on stochastic parameter identi- 
fiability. They stated that Che parameters are identifiable if every 
asymptotically efficient estimator converges to Che true parameters in 
mean square (which is stronger Chan convergence in probability.) 

Wald [20,21] considered the consistency and asymptotic properties of 
Che maximum likelihood estimation sequence. He showed chat under certain 
restrictions on the Joint probability distribution of the observations, 
the maximum likelihood equation has at least one set of roots which is 
a consistent estimate of the unknown parameters. He also showed Chat 
any root of the maximum likelihood equation which is a consistent 
estimate of the parameters is asymptotically efficient. 
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Based mainly on Wald-Kendall-AstrBm theory, Aoki and Yue [l] 
examined the asymptotic properties of the maximum likelihood estimates 
of unknown parsmeters of a class of linear, stable, constant, dlscrete- 
tlsie dynamic systems form where plant noise and observation nolae are 
present. The systems considered by them were restricted to have certain 
canonical structure and were single- Input and single-output. 

For the Identification of linear dynamical systems, Glover and 
Willems [6] established the concept of parametrlzatlon and developed 
sufficient conditions for local and global Identlflablllty from the 

a 

transfer function. Bellman and Astrttm [2] also provided an algorithm- 
oriented least-square identifiability. It can be shown that for single- 
input, zero state systems, local least-square identifiability is 
equivalent to local transfer- function identifiability under some 
assumptions on the least square criterion. The remaining second step 
of the system identification is to identify the system parameter exactly 
from input-output sequences for deterministic systems or to construct a 
consistent estimation sequence by using the constrained maximum likeli- 
hood method for stochastic systems. In both cases, it becomes an 
optimization problem, i.e., we first set a performance criterion which 
is a function of the unknown parameters and then find best estimates 
such that the cost function will reach its minimum. 

In general, the cost function and its derivatives are nonlinear 
and an iterative procedure must be used to find the estimate. The best 
known method of solving a set of simultaneous nonlinear equations in 
which the Increment in each iteration is computed as a linear combination 
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of tho residuals is the Newton method. Kantorovich's Theorm [l2] 
states certain sufficient conditions for the convergence of the Newton 
iteration sequence. However, these sufficient conditions are generally 
highly restrictive end are not easily examined. Moreover, in each 
iteration, we have to calculate the Hessian matrix of the cost function 
which includes the calculation of a bilinear form. These are the two 
main disadvantages of using the Newton method from the point of view of 
practical computation. 

Later researchers have developed some modified versions of the 
Newton method. The most significant for the problem of interest here 
are by Ben-Israel [3] and Pereyra [l33* The main features of both of 
their algorithms are: first, we only need to compute the first order 

approximation of the Hessian matrix of the cost function; second, the 
sufficient conditions for the convergence of the iteration sequence are 
much easier to examine. Although Ben-Israel's algorithm and Pereyra 's 
algorithm are the same if the first order approximation of the Hessian 
matrix of the cost function has full rank, Pereyra 's sufficient con- 
ditions are preferable again from the point of view of practical cal- 
culation [13]. 

In the deterministic system identification problem, the solution 
for the cost function is the true parameter, i.e., the cost function 
will reach its global minimum, zero, if the output sequence generated 
by the estimate matches the measured output sequence. For a known system 
structure, i.e., a given parametrlzatlon, there may be more than one 
isolated point in the parameter space that will generate the same o.<tput 




£ 
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sequence for a given input sequence, i.e., the solution to the identifi- 
cation criterion is generally not unique globally. Herget [7] provided 
a modified version of Pereyrs's theorem and a computation procedure 
employing interval arithaietic to find explicitly the regions centered at 
each local solution in which the solution is unique and hence is locally 
identifiable. The other feature of his work is the use of bilinear 
operators to represent the linear system model. In doing so, the 
Identification problem of linear systems is equivalent to the initial- 
state observation problem of quadratic- in-the-state bilinear systeais. 

This dissertation considers the parameter identification problem of 
general discrete- time, nonlinear, multiple- input/multiple-output dynamic 
systems with Gaussian-white distributed measurement errors. The knowledge 
of the system parametrization is assumed to be known. Concepts of local 
parameter identifiability and local constrained maximum likelihood 
parameter identifiability are established. A set of sufficient conditions 
for the existence of a region of parameter Identifiability is proposed. j 

A computation procedure employing Interval arithmetic is derived for | 

finding the rt.gions of parameter Identifiability. It is shown that if 
the vector of the true parameters is locally constrained maximum likeli- 
hood identifiable, then with probability one, the vector of true parameters 
is a unique maximal point of the maximum likelihood function in the 
region of parameter identifiability and the constrained maximum likeli- 
hood estimation sequence will converge to the true parameters. 


Chapter 2 is a review of Wald's theory [20,2ll on the consistency of 
maximum likelihood estimates, the concepts of parameter identifiability 
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«nd loe«I p«raMC«r ld«atifl«blllty ••tAUslMd by Tm md AntoB [19] «nd 
by Tm [18] «nd the applleatlim of thooo eoneopti to tiio systom Idontl- 
ficotlon probUmt [16]. 

Chaptor 3 it a aurvay of Ballnan and Aatrbh'a [2] dafinltion of 
laaat-aquaM paranaCar idantlfiabilltyf and Glovar and Willan'a [6] con* 
eapta of ayataa paraMtrlaatlon and IdantlfiablllCy fron tha tranafar* 
function. Tha author' a contrlbutlona In thia chapter ara aa followa. 
Sufficient condition for local laaat-aquara paranatar Idantlf lability la 
derived by employing tha conatant rank thaoram [10]. Tha Thaorem of 
Glovar and Ulllama la modified to provide a aufflclent condition for local 
parameter Idantlflablllty of minimal dlmanalonal linear dynamic ayataam 
whoaa Initial atatea are unknownt and a theorem la aatebllahad to ahow 
that for alngle-lnput, aero*atate linear ayatmaa, local laaat-aquara 
parameter Idantlflablllty la equivalent to local parameter Idantlflablllty 
from the tranafer function If aoam conatant rank aaauaqptlona on the Impulae 
reaponae matrix and Ita derlvatlvea are aatiaflad. 

Chapter 4 la a aurvey of the Nawton-Kantorovlch theory [12] on tha 
convergence of the Newton Iteration method, Pereyra'a theory [13] on 
aolvlng nonlinear leeat-aquere problema, t -norma. Interval arithmetic [11], 
end Herget'a reaulta on reglona of parameter Idantlflablllty [7] with 
application to the parameter Identification problem of determlnlatlc 
dynamic ayatema. A numerical example la provided by the author with the 
computer program Hated In the Appendix. 
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Chapter S eontaina cha prlnelpal naw raaulta of thla dlaaartaelon. 

It eonaidara tha paraowtar idantifieatlon problem of fonaral dlaerat«i' 
time multiple- Input/aultlp la-output dynamle ayatama with Gauaalan-whlte 
dlatrlbutad maaaurament arrora. It la mainly a modification of Hargat'a 
raaulta [73 on tha parameter Identification problem of datarmlnlatlc 
ayatama and a ganarallaatlon of Aokl and Yua'a raault [l] on tha parameter 
Identification problem of alngla-lnput/a Ingle- output canonical- form 
linear dynamic ayatama with maaauremant nolaa. A numerical example la 
Included to llluatrate tha computation procedure for finding tha raglona 
of CML parameter Idantlflablllty. 

Chapter 6 glvea the conclualona of thla dlaaertatlon and auggaatlona 


for further reaearch 
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2. MAXZHDM LXIBLZHOOD BSTZMAnOM, PAIAMBTBR 
XOBHTZFIABZLm AHD LOCAL PARAMBTBR lOBMTIFUBILITY 

R«l«v«nt past Invaatlgatlona of b h h cI — llkollhood ••tiaMtion 
paroMtor Idaneifiability, and local paraiaatar Idaatlflablliey arc 
dlacuaaad hora* The flrat raault la A. Wald'a theory [20,21] on the 
aaynptotic propartlaa and th* eonalataney of the naxlnun likelihood 
eatlmata of an unknown peraaMter of a dlacreta proceaa. Then wo will 
diacuaa Taa and Ancon 'a [l9] definition on atochaatic identif lability, 
tha neeaaaary and auffieiant eondiciona for the unknown paraaMtar Co be 
identifiable under aoiaa unifonaity aaauBptiona on eha conditional denaity 
paraaMCriaad by the unknown paraaMtar, and Taa'c [is] dafinition of local 
identifiability. An identification probleai of a claaa of lineer, atabla, 
conatant, diacrate*tiaM, aingle'input/aingle'Output dynamical ayatena 
dlacuaaad by Aoki and Yue [l] will alao be preaented. 

2.1 Preliminary Concepta 

Definition 2.1 

Let X denote an arbitrary nonempty sat. A family of subsets R of X 
is called a sipns field if 

(i) for every AcX, then also A%R where A^ is Che complement of A 

(ii) if Ap A 2 ,»». A^,>— is a countable sequence of elements of R, 

m 

then U A sR, and 

n-1 " 

(iii) psR where 0 denotes the empty set. 

(iv) Elements of R sie called events. 
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PaflnltlOB 2.2 

A probibility Pr. 1« • oMAton ov«r • aMturablA •pae« (X,R): that la, 
Pr. la a raal-valuad function uhieh aaalgna to ovary AtR a niaabar Pr.(A) 
iueh that 

(1) Pr. (A) 2 0 for ovary AcR 

(11) Pr.(X)>l,and 
0 » 

(111) If (a.) Is any countabla union of disjoint avants, than 
n-l 

00 m 

Pr.( U A-) • E Pr.(A ) 
n-l n-l " 

Daflnltlon 2.3 

!--t X ba a sat, R bo s slfsa field and Pr. a probability naasura 
defined on R, than the triplet (X,R,Pr.) Is called a probability space. 
Definition 2.4 

A random variable x la a raal*valuad function whose domain la X and 
which Is R-maasurabla, that Is, for ovary real number X, 

(lOS X|x(uo) sX}sR. 

Definition 2.5 

Let .sCq,"* be a sequence of random variables. If there 

exists a random variable x(u) such that 

11m x-((id -x(ia) for almost all «, wa aay the: 
n-*» 

11m vlth probability one 
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Dtflnltion 2.6 

Let X|,X 2 **’*iXg, •***>» be a sequence of random variables, we ssy the 

sequence {x^} converges to x in probability or converges stocheaticelly 
n»l 

tt> X if 

11m Pr.(|x^(u)) • x((u) I ^e} *0 


is satisfied for every s >0 
Definition 2»7 

00 

A sequence of random variables Is said to converge to x in 

n«l 

the mean square sense if 
2 

(i) E{|x^| }<a) for all n, 

(li) E(|x|^}<oo, and if 

(iii) 11m e{ | x - x^l^) ■ 0 . 
n — 00 

This is written 

l.l.m. * X . 
n-*« 

Definition 2»8 

If X is a random variable, its distribution function is defined 
by 

F^(§) «Pr.[xS§] for all 5 « 

Definition 2.9 

A distribution function F is said to be absolutely continuous if 
there exists a Borel measurable function p over (~«>,oo) such that 
5 

F(5)*J p(t)dt 


for all The function p is called a density of F. 
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D«flnttton 2.10 

Let be random variablea, n^i. The joint dlatributlon 

function of or the distribution function of the random vector 

X* (x^,**«,x^) , is defined to be 

•?„) ■ P'* *?i3) 


where -fl»<gj^<oo, l£l£n. 

Definition 2.11 

Let (x^,\cA} be a family of random variables. They are said to be 
Independent If for every positive Integer n and every n distinct elements 
InA, then 




for all ^sR . 

If • Fx;^ j(? j) for all X^,\jcA< then {xj^.XcA) are said to be 

Independently and Identically distributed. 

Let xj^,X 2 ,—*,x^,--'- be a sequence of random variables with Joint 
probability denstly function p(x^,X 2 i — ,x^; J) , n«l,2, — , which is of 
known functional form but p(xp-— ,x^; 0) depends upon an unknown vector 
of parameters 9 that may have any value in a set This set will be 
called the parameter space. Thus we are confronted with a family of Joint 
probability density functions denoted by {p(Xj,---,Xjjj 9 ): QsO, n»l,2, — }. 
To each value of 0, dcQ, there corresponds one member of the family de- 
noted by (p(Xj,--- ,Xj^; 9 ): n»l,2,*--} which is a sequence of Joint prob* 
ability density functions parametrised by 0. Let (p<x^ ,x^; 9 ^) -. 
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n> 1»2,— } be e eMober of the featly end let be unknown, ^cQ. An 
estinMite of 0^ beeed on the obeervetion sequence n-1,2,— 

is e neesureble function of n>l,2,— , end Is denoted by 

^n*in(Xl» — »*n>» • 

00 

(4^}n_l Is then celled en estlmetion sequence. 

Definition 2.12 

Any estlmetion sequence Cln^n*l -0*^ which converges stochestlcelly 

to Qq Is celled e consistent estlmste for Oq. 

Leswe 2.1.e [The Strong Lew of Lerge Numbers] [ 4] 

Let Xj,X 2 >— , be independent rsndom varlebles such thet 
2 

E X(j"0, E Xj^ «.flo. Let b^^^O converges up to + ®. 

00 7 0 

If E atj^^/bj^ V.00, Chen 

X, +— +x 

Pr.[llm -i -0}-l. 

n-«® bp 

Lenme 2.1.b 

Let x^,X 2 ,— be Independent and Identically distributed 
random variables. 

If E|xj I <«, then 

X, +— +x 

Pr.f lim — — — — ■ e X, ) ■ 1 
n-« n 1 


The concepts introduced abc can be found in references [4], [Sl, and 
[9]. 
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2.2 The Consistency of NexlmuB Likelihood Estimstes 

This section stsnnerizes the theory given by Held in [20,21]* Let 
be a sequence of independent, identically distributed randon 
variables with Joint probability density function p(Zj^,— ,z^; 0), 
n*l,2,— , parametrized by the unknown parameter OcCIcrP, where Cl is the 
parameter space. Let ||*|| be a norm on rP. Let p(z; 0) denote the 
probability density function and F(z,0) denote the corresponding cumu- 
lative distribution function of Zj^^, i.e., F(z;0) «Pr.{Z|^&z}. 

The following assumptions are made. 

Assumption 1 

F(z; 0) is either discrete or is absolutely continuous for all OcO. 

For the next assumption, we introduce the following notation: for 
OcQ and p>0 let p(z; 9,p) be 

p(z; 9,p) ■ sup p(z; 9') 

Il5-5lhp 


For any r>0, let t(x,r) be 


\|t(z,r) ■ sup p(z;9) 

11511 >r 

Furthermore, let 

! p(z;9,p) if p(z;9,p) >1 
1 otherwise 


Similarly, let 

f t(z,r) if ♦(2.«f)> 1 
♦ U.r) - ] 

I 1 otherwise 


Assumption 2 


For sufficiently small p and for sufficiently large r, 

OD ^ 

J log p (z;§,p) dF(z lV <- 
•00 

and 


00 

J log i|*(z,r) dF(Zi§)<® for all dsQ 
•00 


where 0^ is the true parameter point. 

Assumption 3 

If lim 0^ a 0, then lim p(z^ )ap(z;0) for all z except perhaps on a 
i “• 00 i -• 00 

set whose probability measure is zero according to the probability dis- 
tribution corresponding to 0^. 

AssuBiption 4 

If ^ 6 q» then F(z;0j) i*F(z;6^) for at least one value of z. 
Assumption 5 


If lim ll§.|| ~<B, then lim p(z;6.) for every z except perhaps on a 

i “• JO i -*flo 


fixed set whose probability measure is zero according to 
Assumption 6 



00 

j* I log p(z; Oq) I dF(z 
•00 


Assumption 7 

P 

Q is a c losed subset of R . 

Assumption 8 

p(z;6,p) is a measurable function of z for and p>0. 
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t.aMM 2^2 

If then 

E log p(z;0)<Elog p(z; V 

Leana 2.3 

lim E log p(z; 0, p) • E log p(z; 9.) 

p -» 0 “ 

Lenma 2,4 

lim E log i|r(z,r) « -o 

r-*oo 

By the above lemnas, we can prove the following theorems. 

Theorem 2.5 

Let wen be a closed subset of Q. If ^ does not belong to W 

sup (z. »z« ,Zj|j J9) 

0CW 

Pr . [llro I - — - B O} = 1. 

n-® p(zi»Z2»""“»*n’ -0^ 

Proof: By Lenana 2.4 we can choose t^>0 such that 

E log t(z»r ) <E log P(z;®) 

0 

Let be the subset of W such that 
Wj-{9: !l9||srQ. 9zW} 

For each 9«W , we can choose a p„>0 such that 
— 1 


( 2 . 1 ) 

( 2 . 2 ) 

(2.3) 

, then 

(2.4) 

(2.5) 


E log p(z;9,p^)<E log p(z;9^) 


( 2 . 6 ) 


The existence of is gusrsnteed by Lenoiss 2.1 end 2.2. The set is 

closed and bounded; hence it is compect. Thus there exists a finite nuniber 

of points intfj^ such that the union of the spheres with center 

k 

9^ and radiOus i-1, — covers W^. . 

We see that 

k 

OS sup p( 2 i, 22 , — ^ -i* ^ 

^e “i *i 

+ ♦(*iirQ)“*t(z„,rQ). 


Thus we are going to show 


Pr.ilim • 

n 00 


p(gi »ii» Pei)' 

P(2i; Oq) •• 


p^^»ij» PeP 


0} - 1, 1-1 — ,k (2.7) 


and 


« f,. ^zpro) “MUn.ro) ^ 

Pr.ilim — f ^ - 0) - 1 

n - ® P<*i;io^**’ P<*nJf0> 


( 2 . 8 ) 


which is equivalent to showing that 


Pr.ilim Z [ log p( 2 .; Oj, p^^) - log p(z,; e.)]- -®} - 1 

i-1,— ,k 


(2.9) 


and 


n 

Pr.llii^Jjllog - log p( 2 ^; Oq)] = -®) = 1. (2.10) 

But (2.9) and (2.10) follow isnediately from (2.5), (2.6) and the strong 
law of large numbers. 

Theorem 2.6 

Let d (z , ,— ,2 ) be a function of the observations such that 
“n i n 
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in> 

P<*l*-“»*n* V *“ " ***' *“ <2.11) 

Then 

Pr.(lim 4 • 0-} ■ 1. 

n-.«T" “0 
n ^ ® 

Proof: Let ^denote the set of limit points of C^n^n«l* suffices 

to show that for any c >0, 

8up(|l^ - 0 q||: ^@)^S with probability one (2.12) 

Suppose that there exists a 4e(^such that ||id-0^||>c, then 
sup p(z,, — ,*„;«)^p(*i, — 

ll«-»oll** 1 *« « 

for Infinitely many n. But this implies 


sup p(zi “.z_; 6) 


p<*l» — »*n’io) 


(2.13) 


for infinitely many n by (2.11). 

By Theorem 2.5, (2.13) is an event with probability zero, thus (2.12) 
holds with probability one. We recall that the maximum likelihood esti- 
mate 4^ is obtained by 


p(Zl,—“ P<* 1» — ’*n’-^ for all n (2.14) 

A 

If exists, then 

P(^1,”“»S» in) 

P(Zj. 


•*n*io> 


^ 1 for all n and for all x. -,x . 

1’ n 
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By Theorem 2.6, ehe maxinnmi likelihood estimete Is conslscent. 

2.3 Parsmeter Identif lability and 
Local Parameter Identifiabllity 

This section sunmariaes the work by Tse end Anton in Ci9] and by 
Tse in [l8l. 

CO 

Let (z } be a sequence of observation statistics with joint 
n n“l 

probability density function p(Zj,— ,z„*» 8) , n*l,2,-— , parametrized 
by the unknown parameter where Q is the parameter space which is 

a compact subset of R^. Let ||*|| be a norm on R^. The true parameter 8^ 
is known to lie in the interior of The parameter identification 

problem is to estimate the true parameter 8^ based on the observation 

00 

sequence 
Definition 2.13 

The parameter 8^ is said to be identifiable if there exists a sequence 

A CO 

of estimates (8 } which is consistent in probability, i.e., for any 6, 
n'n»l 

e arbitrarily small, there exists an N(6,e) such that for n>N(6,c) 

Pr.Cll4^-8Q||>6}<s. (2.15) 

For brevity, we let 

p(Z^; 8) ■ p(z^,— ,z„;8) , (2.16) 

and 

p(Zn|Zn.i; 8) - p(Zn; 8)/p(Zjj_i; 8) forn»l,2,— (2.17) 


For ten and p > 0, let 
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The following assumptions are made* 

Assumption 1 

p(Zq, 0) is measureble in with respect to pCZ^^: Jq) dZ^ and contin- 
uous in 9cn for Z,^ almost everywhere^ i.e«, for s>0 and ^0, there exists 
a 6(c) >0 such that for all O'cO with ||9* i'||<6 we have |p<Z^; J) - 
p(Z|,;9^)|<c for Z„ alaiost everywhere* 

Assumption 2 


J „ log P(*n»pl^n-1’-^ P^^n’-O^ 

and gcO 


and 


r log P(«„|z„.l!So> P«n'So) ““n*- 

K 

for all n ■ l,2g— 

Assumption 3 

For all 9c0 and some Pq>0, 

o 2 

Var.{j^E^ log p(Z|j,p|z^.i; 9)) - 0(n ) 

for all OXpXpQ, where O(n^) is such that 

n“* SB n 

Assiaaption 4 

Let the set B (9) be 
n ^ 


(2.19) 


(2.20) 


( 2 . 21 ) 


B^(9) m (Z„: p(Z^; 9) -0} 
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then for oil 0 ^f 02 t 0 . wo hovo 

for n- 1,2,— (2.22) 

Since Che only Informetion about la the obaervation aequenee atatlatiea 

CD 

with their Joint denalty function p(Zq; 0), n«l,2,— , If there 


are two points 9j^,02<()> 


P<2nJ5l) ■ P<2n;52> 

(2.23) 

or 


P<*n|2n-iJ5l> ■ P(*nl2n-l'i2^ “ 

(2.24) 


we are not able to distinguish 0^ and ^2 
Definition 2.14 

Two parameters 0^,02<n, Ji <^^2 unresolvable If the 

equality 

P<*nl2n-l5S2^ <2*25) 

holds with probability one for all except s finite number of integers 
n>0, i.e., if (2.25) holds with respect to the measure P(ZgiO^)dZ^ as 
well as p(Zg; ^ 2 ) 

Definition 2.15 

The set 0 is said to be identifiable if no two elements in Q are 
unresolvable. 

By using the constrained maximum likelihood method, the identification 
problem is: find 6^*0 as an estimate of 9^ such that 

P<2„jia' " gg P<2„; 9). n-1,2, — 


(2.26) 
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Since 0 !• eonpcet and hcncc is closed end bounded, and p(Z_; 9) ia 

n •• 

continuous in 9 by assuaipcion 1, a solution to (2.26) exists and the 
estiaate sequence * consistent estlswte for if is unique. 

This follows from Theorwn 2.6. Bowever, if there are two potota 

^1 **- 2 * *“®** ^**** 

lin p(Z ;5.) • lia P(Z ;5-) (2.27) 

n— " ‘ n— " ^ 

then it ia obvious that Ko converge. 

Definition 2.16 

Two pai laeters 9j^,92«n, are said to be CKL unresolvuble If 

lia P(*„^*n-l’-l^ " *’^*n^^n-l’-2^ (2*28) 

n "• 00 ” n ^00 

with probability one. 

Definition 2.17 

The set 0 is said to be CML identifiable if no two aleaents in 0 are 
CML unresolvable. 

The following theorea was given by Tse and Anton in Cl9]. 

Theorea 2.7 

If for all exists an countably infinite set 

LCI*^ (I'^athe set of positive integers) such that 

frith nonxero probability with respect to 9^ and 9^ unifomly in ntL, then 
0 is CML identifiable. 
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Tha abova eoneapta on paraMtar Idantlflabilltp can ba applla4 to 
tha fjataa Idaatlfieatlon problaa. 

Conaidar a llaaar dtacrata-tlM ayataa daacribad by: 


x(k+ 1) - l^(k) + C^(k) 
jOO -llx(k) + 2 f(k) 


(2.29) 


iriiara F la tha (n x n) atata tranaltion matrix 
G la tha (nxq) Input natrlx 
H la tha (r x n) output natrlx 
x(k) la tha n>atata vaetor 
a(k) la tha foutput vactor 
u(k) la tha q- Input vactor 

v(k) la a Gauaalan white nolaa with aaro naan and eonvarlanca natrlx Q. 

Lot tha Initial atata x(0) Tha paranatar la to 

D P 

ba Idantlflad. Ua aaauna G^aOCR*^ whara 0 la a conpact aubaat of 

Furthamora. wa aaauna that 

(1) tha ayatan la atabla for all MO. 

(2) tha ayatan la conplataly controllabla. l.a., 
rank[G.FG,— ,F^ ^G]*n for all GaO, and 

(3) tha a/atam la conplataly obaarvabla, l.a, 
rank[H’^^,(HF)^, — ,(HF"“b^]-r. for all 

Tha aaaunptlon cf controllability and obaarvablllty inpllaa that tha ayatan 
la of nlnlnal dlnanalotf and aqulvalant ayatana for (2.29) axlat. The 
Joint probability danalty function of dia obaarvatlon aaquanea (a(0),— 

— .1,(M))» M •0,1,2, — , la glvan by 
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p<s(0) . — «J00 ;•> ^^oPv(k) 

N 

- Cofift. «cp(-l £ Cj(k) -%(k)]V^[j(k) -%(!(> 3) 

2M) (2,30) 

wh«M x<k) U eh* •olution to (2.29) for « i^oiXi gtCi, Tho CML ootlBation 
Mthod i« than to find •• on oitiaoto of ^ ouch tiuit 

P(S(0)» — ^(M); jb - Max p(s(0),— ,a(M); •) 

M-0,1,2, — (2.31) 

If thara ara ganarata tha aana ^(k) 

aaquanea whan appllad with a givan input aaquanca, than 

M-0.1,2, — (2.32) 

for tha aaaa naaauranant nolaa dlatrlbution. Thuf by daflnitlona 2.14 and 
2.16, 0| *(^^2 unraaolvabla and CML unraaolvabla. Slnca ayatan 

(2.29) la Blniaal. tha following thaoran, which providaa tha aufficiant 
condition for unraaolvability, followa iaaadiataly. 

Thaoran 2.8 

5l • (2^)i»?l»Ci»*i»Ql} 

®2 " i5()2’^2’^2’^2*^2^' 5l^52* 

0^ and J 2 koth unraaolvabla and CML unraaolvabla if thara axiata a 
nonaingular. nonidantity natrix P aueh that 
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Pl-PFjP'^ 

G1-PG2 

i5l0“f520 

Q 1 -Q 2 

Proof: Let the state vectors generated hy 0^ and Q 2 ^ 2 ^^) 

respectively. Then Xj (k) • pX 2 W for any 

input, end the two systems paramet'^ ized by 0^^ and 0^ respectively are 
equivalent. 

Therefore 


1 M T -1 

« Const. • exp {-^j^yz(k) - HjXj^(k) [z(k) -H^x^Ck)]} 

M 

« Const.* exp {• ■2 k^ 3^Q2’^C5.(’^) - H2*2(k)]} 
“k2o**v(k) -« 2 X 2 (k); ©z)* M- 0,1,2,— 


Hence 9, and 9 are both unresolvable and CML unresolvable. The above 
_1 _2 

theorem is a modification of the one given by Tse and Anton in [l9]. 

To illustrate Theorem 2.8, we have the following example. Consider 
the system 


jc(k+ 1) 


9j 1.0 


0 9, 


x(k) + 


0 

9. 


u(k) 


(2.34) 


_z(k) ■[ I 0 ] x(k) + v(k) 
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x(0) 




2j 


with 9* [O^ 92 9^ 9^ 9^] to be identified. 

By Theorem 2.8, the following etiuetions are obtained 


®3 

1.0 

0 

«4 

0 




[1 

o ; 

■« i ' 

m 

«2 

1 


'11 


■^12 


P P 
21 *^22 


9. 


'u 

’12 

'^21 

**22 


0 

9. 


^1 

'l2 

P 

P 

21 

22 



^4 

9 

el 

.«2, 


1.0 


9. 


'11 


'11 

^12 

'21 

^22 


'12 


P P 
*^21 *^22 


-1 


and the solutions are 


P, 1-1.0 


Pj2-0 


‘21 


«3’®4 


P22-1.0 

9 '.93 

»l'-«l 

®^«2^*’21 ®1 


(2.35) 
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8ptelfic«lly» if VC let 


0.1, 0*3, 0.7, 1.0], then 
ij-CO.S, -O.l, 0.7, 0.3, 1.0] 


P« 


and 

' 1.0 0 

,-0.4 1.0, 

I.C., the follovring two systems 


and 



0.3 I.O' 


0 

x(k+l) - 


x(k) + 



0 0.7 

» t 


1.0 

k i 


u(k) 


s(k) ■ [ I 0 ] x(k) + v(k) 
x(0) « [0.5 , 0.1]^ 



o 

• 

e 

o 


» * 
0 

x(k+ 1) ■ 


x(k) + 



0 0.3 


1.0 


u(k) 


s(k) ■ C 1 0 ] x(k) + v(k) 

x(0) - [0.5 , -0.1]’^ 


(2.36) 


(2.37) 


are unresolved in any compact subset of containing 0^ and 

If there is more than one vector of parameters in Q that will generate 
the same observation sequence Joint density function, the parameters are 
not globally identifiable. However, if there exist regions around each 
point and if there exists a local estimation sequence in each region, we 
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n 


are still able to identify those parameters by some identification algor* 
ithms of local variation type. The following concept is established by 
Tse in [18]. 

Definition 2.18 

The parameter OqSO is said to be locally identifiable if 
(i) there exists an open set such that 0^ is an interior point of 
Sq, and 

(ii) there exists a consistent estimate 



in Sq where 


is the closure of S^. 


We will call Sg the region of parameter identifiabllity. By imposing 
the same assumptions as the above on the Joint density function of the 
observation sequence, Tse [18] has the following theorem. 

Theorem 2.9 

2 

If for all n"l,2,— , there exists a X >0 such that 

^ 1 ) 

> X^I (2.38) 

pxp 


where represents the expectation with respect to the density function 
p(Zn'. dg). then 9^ is locally identifiable. 

Another weaker sufficient condition for local identifiabllity was 
also established. Define 




(0)^E, 


i. 10, P,., (9) a log 

89 89 ^ ‘ 


(2.39) 


L 




Si 


A 







Noting that 


log Pv ,(S) „a ipg Pk.i<S>,r® ■’k.i® 


•)--Egt[ — 8j 


3 t 


- v A '' IT"' 


j .a iogPi.i<«), 


- - S J. . («) 
i-k 


We have 




■’l,n«0>-l5l-'l.l«0’ 


Theorem 2.10 


If there exists a \ >0 such that 


n “* 00 n“*® 


then is locally identifiable. 
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Daflttltlon 2.19 

A subset SCR^ Is said to be locally Identifiable If all the elements 
In S are locally Identifiable. 

Theorem 2.11 

A sufficient condition for a subset SCR^ to be locally identifiable 
is that 

lim Ji^n(g) X^(g) >0 for all 9sS (2.46) 

n“*oo ’ 

In the next section we will present a system Identification example 
given by Aoki and Yue [1]. 

2 .4 An Example of S;ystem Identification 

Aoki and Yue [ 1] examined the asymptotic properties of the constrained 
maximum likelihood estimate of the unknown parameters of a class of linear, 
stable, constant, discrete- time systems with observation and plant noise. 
The system considered by them is in the completely observable conpanion 
form and is single- input and single- output, hence the system representa- 
tion is unique. Therefore local identif lability will imply global identi- 
fiability for the class of systems considered by them. It is 
obvious that global identifiabllity implies local identiflabillty. 

Consider the dynamic system represented by 


x(k + 1) ■ Fx(k) + Gu(k) 

y(k) ■ Ite(k) 

z(k) *lU(k) + v(k) 


(2.47) 
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where F Is en (nxn) netrlx, 6 is en n-eonponent vector, H>[10— 0] Is 
a (Ixn) matrix, and v(k) is a seqtienee of Independent and identically 
distributed random variables with zero mean and finite variance o^, i.e., 
v(k) ~ N(0,o^), k« 0,1,2,—. F has the following completely controllable 
companion form: 


— 1 0 • e • 0 

"ft 0 1 0 * • 0 

r* a eeaeee 

a e a e e e a 

a a a e a a 1 

■•fl 0 a a a e 0 

n 

la 

T 

and G-Cbj.b^, — ,b^] 

The initial state jc(0) is an unknown n-vector. Our purpose is to 
identify 

T 

0* [oi »“"•»*„» * a 2n-vector, 

and the unknown initial state x(0)axQ. 

The input sequence u(k), k •0,1,2, — is known and is assumed to be 
uniformly bounded. 

Suppose we take M observations. Define 


U„-[u(0), u(l),— , u(M- 1)J^ 
Vjj*[v(0), v(l),— , v(M-l' j‘ 
Xj,-[y(0), y(i),— , y(M-i)f 
Zj^»[z(0), z(l),— , z(M-l)]^ 


(2.48) 
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Th«n 


Vm - "iAi ♦ 


(2.49) 


where 


M MXM i-1 1 


“m "AV 

I, 


nxn 


■(---- ) (2.50) 

®M-n,n 

and S is the MxM shift matrix with element Another way to 

eiqiress the input-output relation of (2.47) is as follows: 


Yjj - H^+ Ej^(O) (2.51) 

where 

a>, - <-SJ£„. -S^Ih,— .- s\, ,8%,) (2.52) 

which is an MX 2n matrix. 

We assume that the true parameter 9^ is an interior point of 0 where 

2n 

0 is a cogqiact subset of R . Furthermore, we assume that the system 
(2.47) is stable for every 9«n. By (2.47) and (2.49), the output sequence 
can be expressed as: 

< 2 - 53 ) 

and the joint probability density function of is 
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Const.*exp(lU|||^- (2.54) 
2 a 

The eonstrelned likelihood estimate of j end Xq, denoted hy ^ and x^ 
respectively, are obtained by 

logp<Zj-;i^,i )- Max log p(Z ; 0 ,iIq) (2.55) 

-M 9cn,x^,R« M ^ 

For any 9cQ, Max log p(^; 0*2E^) is obtained by 
*■ *^anR“ ^ 

?0M<» ■ C'm'' V V W <*•«> 

Then 4 is obtained by 

- (2-57) 

and 



(2.58) 


The following theorems were given by Aoki and Yue in [l]. 

Theorem 2.12 

If the system (2.47) is completely controllable, and G is not a zero 
vector, then the constrained maximum likelihood estimate 4 converges to 
the true parameter ^ with probability one if and only if 



where 


^.2n ■ iM-— •s\> 


(2.59) 


33 


Thaerni 2.13 

If the eystcm (2.47) ie completely controllable » and G la not a aero 

vector t then the conetralned maximum likelihood eatimate 4 convergea to 

"Ti 

the true parameter ^ if and only if 

We note that the positive definiteness of the two matrices in Theorem 12 
and 13 are sensitive to the input sequence therefore, input synthesis 
is an inq>ortant factor for the identif lability of the system parameters. 
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3. LOCAL LEAST-SQUARE PARAMETER IDENTIFIABILITY 
AMD LOCAL PARAMETER lOEMTIFIABILZTT FROM THE 
TRANSFER FUNCTIOM OF LINEAR DYNAMIC SYSTEMS 

In this chapctr we will discuss speeifieelly the peraneter 
identifiebility of lineer dynesdLc systesw. He will establish the concept 
of parameter identifiebility without considering the identification 
algorithm, the algorithm-oriented least-square identifiebility [2J, and 
the transfer- function identifiebility for linear dynamic systems [6]. 
Furthermore we will discuss the relation between the local least-square 
identifiebility and the local transfer- function identifiebility. 

3.1 Notation 

He will present specifically the manipulation of the bilinear 
operators in this section which is necessary for the approach later on. 

For a real-valued function of DcOCR^, i.e., if if(9) 

is differentiable with respect to DsCl, we define 


where 

i • C«1.— .«pf. 


If is an m-component vector- valued function of OsO, we define 
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•nd 




• st.W 



l'<» • 



1 - 1 .— »■ 
J ■ I,— »P 
k- 1»— »P 


which Is an (mxpxp) blllMsur operator [l4]. For A^O) an (nxp) matrix, 
A(9) ■Ca^j( 8)], wa daflno 


A'(«) m 



1 • 

J ■ I,— ,p 
k» 1,— ,p 


which Is an (m x p x p) bilinear operator. If B Is 
operator , then the product of B with a p-vect'*r 9 
whose 1,J elosent Is 


an (mxpxp) bilinear 
la an (mxp) matrix 


(B9) 


IJ 


P 

k-1 iJk k 


Moreover, 


B99 Is a m-vector defined 
B99 - (Be)f 




by 


We denote the permutation of B as 


* 

B , where 
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B It ttld to bo tjtBMtrle If 
B«B* 

T 

Tbo erantpoto of B It doaotod bp B , ifhort 

Tho product of t bilinotr oporttor B with t (p x q) nttrlx it on (u x p x q) 
billttotr aboao l»J»k oloMOt la 






Tho product of a (qxa) ootrlx with on (mxpxp) blllnoar oporttor la at 
(qxpxp) blllnoar oporttor whooo l»j»k oloaont la 

^^*^lJk“A^U®tJk 


Glvon two notrlcoa 


A-Ctij] , l,J-l, — ,n , and 
B*Cb£j3 $ i»J"i»“”»® • 

wo fom 0 now nttrlx C with oloaonta c . obtained bp aultlplplng otch 

1 J a KC 

oloaont of A bp otch oloaont of B In tho following wop: 

Horo, tho pair of Intogora (k,j) act aa the flrat Indox, and tho 
pair of Intogora (k,t) act at tho aocond Indox, wharo 


l,k* 1.2, — ,n, 
J.t ■ l.2,*«“.a» 
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T^.« matrix C la eallad tha Irottaekar product [19] of tha matrieaa A and 
B and la danotad by 


C-A®B 


For oxaB^la, lot A and B ba aaeond ordar matrieaa. Than thalr Kronaekar 
product la a fourth order matrix, trtileh can ba urlttan aa 




•ll‘*l2 


*12^12 

c • 

*11^21 

•ll**22 

•l2**2l 

**12^22 


•2l**ll 

*2l**l2 

*22**! 1 

*22^2 


•2lhl 

m 

■2l**22 

•22*^21 

*22**2l 



®11;12 

®U;2l 

®ll;22 

m 

®12;ll 

®12;12 

*12;21 

®12;22 


®21;ll 

®21;12 

®2l;2l 

®2l;22 


®22;ll 

®22;12 

®22;2l 

®22;22 


For an (nxm) matrix A-Ca^^j], A la daflnad by 

tel 


A 


nl 

‘l2 

. 1 . 
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3.2 Local Least'Square Parameter Identlf lability 
A general contlnuoua-tlme deterministic dynamic system Is described 
by 

x(t) -♦(x(t) , u(t) , t; 0) 

^(t) -|(x(t), u(t), t; «) (3.1) 

w' d -e x(c) Is the n>state vector, u(t) is the q«lnput vector, j;(t) is the 
r-outpuc vector, ts[0,oo) ■ r"*", x R** X r"*" xO~*r”, and g:R** x R** X R xfl“*R^. 

p 

0(0 CR is the unknown parameter to be Identified. If the system Is dis- 
crete-time, then it is described by 

x(k+ 1) - i|rQc(k) , u(k) , k; 0) 

i(k) -i(x(k), u(k), k; 0) (3.2) 

k-0,1,2,— 

p 

We assume that 0 is a compact subset of R and the system is stable 
for all ^0. We must note here that 0 may stand for the system model 
coefficients only, e.g., the F, G, H matrices of the linear systems, or 
it may Include the unknown initial state. To distinguish the above two 
cases, we have the following definitions. Let A denote the set of all 
admissible inputs, let h(t;0) or h(k;0) denotes the output generated by 
9 when applied with a 
Definition 3.1. a 

Let 9 stand for the system model coefficients only, then for the 
continuous -time systems, 9^ are said to be unresolvable in 


n if 
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h(t;5i) -h(t;i2) 

for all ucA» and CcR^. 

For the discrete- time systems » 9^^ and ^2 unresolvable 

in n if 


h(k;9j) - h(k;i2) 

for all i!<A) end kcl^. 

Definition 3.1.b 

If 9 includes the unknown Initial state then for the continuous- 

_o* 

time systems, 9^ ,9^00, 9]^f^92 are sdid to be unresolvable in 0 if 
«h(t;92) 

for all ucA and tsR*'. 

For the discrete- time systems, 9^ and 9 ^ are said to be unresolvable 
if 

for alljfcAand ksl^. 

Definition 3,2 

A parameter ^cO is said to be locally identifiable if there exists 
an open sphere S(9^,p) with radius p >0 centered at 9^ such that there is 
no other 9cS(9q,p) DO, 9^9^ which is tinresolvable from 9^. 

The above definition is made independent of the method for recovering 
9^. However, Bellman and Astrom £2] established an algorithm-oriented 
definition which is called the least-square identifiabillty. Specifically, 
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they first set e least-squere type criterion parametrized by the unknown 
parameters and sought the unique local (or global) minimum of the criterion. 
Parameter Identlflabllity was then Implied by the uniqueness of the 
minimum of the criterion. 

Consider the criterion given by 
T 2 

J (0) a r |lh(t;0) -2(0 II dt, T>0 (3.3) 

for the continuous-time system, or 

Jj,(e) ajj|h(k;0) -2(k)|!^. M-0,1, — (3.4) 

for Che discrete-time system. ||*|| denotes the norm, ^(h) are 

the measured outputs of Che continuous- time systems and the discrete- 
time systems, respectively. The following definition was given by 
Bellman and AsCrom in [2]. 

Definition 3.3 

Let Oq be the true parameter of a control system parametrized by the 
unknown parameter 9. Then 9^ Is said to be locally least- square identi- 
fiable if the criterion J^(9) or Jjf(9) has a local minimum at If 

Che minimum is global, 9 q Is said to be globally identifiable. 

To establish Che sufficient conditions for 9^ to be (locally) identifiable, 
we proceed as follows. We consider the discrete- time system. To find 
the local minimum of we take the derivative of J^(9) with respect 

to 9 if J,.(9) is a continuously differentiable function of 9. Since J„(9) 
is a scalar, JjJ(9) is a p-component vector function and is a (p x p) 

matrix function. If 9^ is the true parameter, then h(k;9Q) 
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k«0»l,— , and . Sine* ^(§) i« noniMgatlva, ^ 

la a mlnlnal point of *nd hence J^(Sq) "0. However* If is to 

be locally Identifiable, 4 nnist be the unique nlniaal point for J„($) in 

some neighborhood of i.e., there must exist an open sphere S(§^,p)cn 

with radius p>0 centered at 0^ such that if OcS(Oq,p), then 

j'(0) ytO and hence % is not a minimal point of J (§). If we can establish | 

a sufficient condition such that J'(§) is an injective function (a one-to- 

one mapping) in some nei^borhood of 0^, then this condition will imply 

that is locally Identifiable. We first state a result given by 

Narasimhan in [l0]< 

Lenma 3.1 | 

Let S be an open set in R*^ and be a C mapping (a k- times 

continuously differentiable function) «rith k^l. Then if i|t'(0) has 

j 

constant rank J In a neighborhood of OqCS, i|r is locally injective at 0^ 
if and only if J*p. 

Theorem 3.2 

A sufficient condition for to be locally identifiable is that ] 

there exists an open sphere S(9^,p)cQ with radius p>0 centered at 9^ 
such that the (p X p) matrix J^(§) is nonsingular for all 9cS(9q,p). 

Proof: Since >^(§) is nonsingular for all 9 sS(0q,p), it has constant 

rank p for all JsS(9^,p). By Lenna 3.1 i^ locally injective on 

S<§Q,p). Let «sS(§Q,p), 9!‘eQ, then J^(§)^0 hence Jj^(9)^0, M- 0,1,2,. 

Thus 0^ is the unique minimal point for J|^(§), M* 0,1,2,— , in S(§^,p). 

The above theorem is an immediate result of Lemma 3.1. The least- 
square identifiability and the identification algorithm for finding the 
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region of parameter Identifiabllity will be studied extensively In 
Chapter 4* 

3.3 Local Parameter Identifiabllity from the Transfer Function 

In this section the systems considered are dlscrete>tlme. We will 
first briefly introduce the realization theory established by Ho and 
Kalman In [ 8] and explain the distinction between the realization and 
Identification. Specifically, the parameter identification from the 
transfer function can be viewed as the realization from the transfer 
function restricted to the given parametrization. 

The quadruplet (f, G, H, D} defines the internal description of a 
system, which we shall denote by Z, via the equations: 

x(k+l) -Fi(k) + C^(k) 

^(k) «Ite(k) Du(k) (3.6) 

where F is the n x n state transition matrix 
G is the n X q input matrix 
H is the r X n output matrix 
D is the r X q direct-coupling matrix 
x(k) is the n-state vector 
^(k) is Che r-output vector 
u(k) is the q- input vector 

The external description of the system I is the zero-state impulsive 
response description, namely the description in terms of an impulse input 
and the corresponding output. There are two ways to represent Che external 


description of I. One of them is the time doissin description. From 
equstion (3.6) , it csn be essily seen thst the impulse response of £ is 
given by 

Wq - D 

- HG + D 


M 


M-1 . 

C S HF^G} 
i-1 


+ D 


(3.7) 


snd the impulse response mstrix is given by 


^ m 

“o 


• s 

HG*^ + D 

“l 

e 


HFG -I- HG D 
• 

• 

s 

Wm 

• 

m 

• 

M-l* 

[^Z^HF^G} + D 

e 

• 

• 

» « 


e 

e 

• « 


(3.8) 


By the frequency domsin description, the input is relsted to the output by 
the trsnsfer function T(z) such that 


Y(z) - T(z)U(z) 


where 


-1 

T(z) - (h(zI-F) G+D} 


(3.9) 


(3.10) 


zcC (the field of complex numbers). 




iBi'Kiil. 
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Through the concept of realisation, the external description of a 
system can be related to the Internal description of a system. The prob- 
lem of realisation can thus be stated as follous [8]: construct 

such that the Identity (3.7) holds, l.e., given a sequence of constant 

OP 

(rxq) matrices ^^nd a quadruplet {f,G,H,d} of constant matrices 

such that 



- D 

• HG 


«M 


M-1 

HF G 


The sequence Is called Che Markov parameters of the system E. 

The dimension of E Is defined by 

dlm(D > dlm(F) 

We say that the realization [F,G,H,d} Is minimal If the dimension of 
F Is less Chan or equal to the dimension of any other realization of E. 
From Che linear system theory, we know that a realization Is minimal If 
and only If E Is both completely controllable and completely observable, 
l.e. , If and only If 

rank[c,FG, — - , f” ^G] ■ n (completely controllable) 

_ T T n* 1 T 

and rank[H ,(HF) , (HF ) ] * n (completely observable) 
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Furthermore, given an external description, two minimal realisations 
Zl ■ CFi,Gi,Hi,Di) and Z 2 • {F 2 .G 2 *H 2 ,D 2 ) are equivalent if and only if 
there exists a nonsingular nxn matrix P such that 


F2 - PF^P 


-1 


PG, 


H2 - HjP 


D 2 -D 1 


(3.9) 


These two equivalent Internal descriptions differ only in the co- 
ordination of their state spaces. 

Even though we have the knowledge of the external description and 
the miniaial dimension of I, generally we are not able to determine the 
quadruplet {F,G,H,D} uniquely unless the structure of {F,G,H,D} is con- 
strained in some specified form. We can illustrate the above statement 
by the following exanqple [2]. 

Consider 



» • 
-(Oj + 02) «3 


1 

x(k+ 1) ■ 


x(k) + 



®2 -<®3 + V 


0 


y(k) - [ 1 0 ] x(k) 


(3.10) 


The transfer function of (3.10) is 


•“2 


z 


z O 3 0 ^ 

+ z(e^ + 02 + ®3-»’94) + (Oi + « 2)(«3 + V " ^2®3 


T(z) 


(3.11) 


A6 


Thus we have only three equations to solve the fou** unimoims 

«3 + ®4 - Sj 

91 + 02 + 93+ 9^ - S2 

(9i + 92)(93 + 94) - 9393 - O3 (3.12) 


where a^, 83 » and are known constants. For this under determined set 
of equations, one of the four unknowns has to be dependent on the other 
three, hence the representation of (3.10) is not unique. However, if the 
system representation is constrained to have the canonical form: 


■ 

0 1 


‘ «3 

x(k+l) - 


x(k) + 



9i 93 


®4 

» 1 


u(k) 


y(k) -Cl 0 ] x(k) 


then 


9 o 2 + 9 a • 

T(z) - —X 

- 932 - 9j 


sjz + a^ 


z +832 + 8^ 


and 


(3.13) 


(3.14) 


9, 


9, 


9, 


«4 * *2‘*3 


(3.15) 


is the unique representation for system (3.13). The identification 
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(realization) a£ ayatam (3.13) ia an axaapla of the canonical paraaatar 
identification problem which haa bean thoroughly etudied. By canonical 
parameter identification, we mean finding a cartain set of parameter in 
a given canonical paremetrization which when applied with the input 
sequence from a given set of input>output sequence will generate a set of 
output data which will match the given output sequence within some well 
defined degree of accuracy* The primary reason for finding canonical 
parameters is to obtain a model Which gives a good match to the measitred 
input'Output data. However, it may not be desirable to use a canonical 
form for a given physical system. That is, it may be desirable to Identify 
specified parameters in a given parametrizatlon. Hence the parameters 
identified in a specified canonical form may have little or no recogniz- 
able relationship to desired physical paraaieters. To expound the above 
statement, we give the following example. 

Given the frequency domain external description of a second order 
zero-state system I: 


T(z) - -j— ^ 

z + az + b 

The canonical parametrizatlon: 



0 

1 


» « 
0 

x(k+l) - 

«1 

m 

x(k) + 


y(k) - C 

1 

0 ] 

x(k) 



u(k) 


(3.16) 


(3.17) 
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can ba uniquely datamlnad by 


®l--b 

«3-c 


(3.18) 


hanee eha canonical paraMtriaatlon Is globally Identifiable. However, 
If we are given the phyalcal parametrlsaclon which la the saise as the 
example In section 2.3: 



» 4 

9^ 1.0 


• * 

0 

i(k+l) • 


x(k) + 



0.0 9- 

» “a 


. *3. 


u(k) 


(3.19) 


y(k) -Cl 0] x(k) 
then we get the following set of equations: 


9^+82 — “S 


® 1®2 “ ** 
93-0 


(3.20) 


It is obvious chat 9^^ and 82 can be interchanged without affecting the 
transfer function. Hence this physical parametrlzation is not globally 
identifiable but only locally identifiable since Che two parameters: 


9j — (9^, 82,9^) and ^2 — (92,9^,83) 


will both generate the same transfer function even though they are isolated 
in the parameter space if 8^ ^^ 2 ’ 
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A paraMtrlzatlon of cho syottn Mtrleo* (F»G,I1,D} It a eontlnuoualy 
dlfferantiabla function which naptH, the paratMtar apaco, to tha 
^n(n<f q*f r)<frq ^ paranatri cation ia a c' function 


(P,G,H,D)(9):ncR** 


n(n + q + r)+rq 

R 


For brevity, we let n(n-f q*** rHrq "t* Tha above definition waa given by 
Glover and Uillema in [6]. 

Definition 3.5 

Given e paraawtrlcation of a ayateoi £, two parametera 9^,92ant ^^2 

are aaid to be uoreaolvable from the tranafer function if 

H(gi)(cI-F(gp)“^G(§j)+D<gj) - H<«2)(*I-F<S2))‘^CCfi2)+D<fi2^ 

(3.21) 

for all zcC and z (\(F(G^)) , \(F(§2))) where X(*'j denotea the eigenvaluea 
of the correaponding matrix, or equivalently, 


D<Sl> • D(«2^ 

H(Si)F^(G^)G(§i) - H(«2)F^(«2^G(S2) 1-0,1, 2.— (3.22) 

The following definition la aimilar to the one given by Glover and 
Wlllema in [ 6]. 

Definition 3.6 

A parataetrizatlon la said to be locally Identifiable from the trmafer 
function at OqCCI If there la an open aphere S(§^,p) c(l with radlua p >0 
such that there la no 4cS(9^,p), 0^^, which la unreaolvable from ^ l.e.. 
If there la a ^S(0 q,p) such that 


I 

i 


so 



1 


I 



D(*> • D(S^ 

H(S)r^(S>G<s) • R(Sg)r*<Sg>G(S^, t-i,2, — O.M) 

than 

By Lana 3.1, an ianadlata raault follows. 

Thooran 3.3 

Lac (F,G,H,D)(0) : O-'lK’ bo a paraswcrlsatlon of Z, Chan Cha 
paraswcrlsaclon Is locally Idandfiabla from Cha cransfar funecion ac 
Ogsn if Cha gradlanc of cha Markov paramacac oncrlx M^) wich raspacc Co 
0 has conscanc rank p in an opan sphare S(^,p) wlch radiua p >0 eancarad 
ac Oq whara 

’ D(«) 

H(0)G(0) 

M(G) - H(g)F(g)G(0) 

a 

H^«)f”(0)G(G) 

a 
a 
a 
a 

If a paraawcrlsed syscam is of minimal dlmsnslon, Chan 1C Is ralaced 
to Its zero-scaca and zero-input equivalent systems by similarity trans- 
formations. For the physical parameter Identification problem, we are 
Interested In Che equivalent sysCeiu which have Che same parasMCrlzatlon, 
l.e., we wish to Invascigate if rhere Is any transformation matrix which 
will transform a parametrized system to an equivalent system with 
different parameter values but with same parametrlzatlon. Specifically, 


SI 




I 


¥m thall Iftvtttclgat* th« tolutlon (P,^), P«QL<n)» th« •pact of nonolngular 
(nxn) matrleoa, jcO, of cho following aot of •quaclono: 

PP(8)P*^ - P(J^ 

PO(g) - G(^ 

H(S)P'^ - 

»<S) - D(So> 0»2^) 


whoro i« tho truo paraawtor. It is obvious that if thara it an opan 
aphara S(^,p)cn auch that ia tha uniquo solution of (3.24) in 

GL(n) x8(^,p)» than tha paraaMtrisation it locally idantifiabla from tha 
transfer function at 9^. Tho following thaoraai provides tha sufficient 
condition for (3.24) to have unique solution locally which was given by 
Glover and Wiliams in [6]. 

Thaoran 3.4 

Let {F,G,H,D}: be a given parametrisation of the system matrices 

{f,G,H,D} and suppose [F,G,H,d} is minimal. 

Ut 


e(P.G) 


PF(£)F 

PG(fi) 

H(S)P"^ 

D(fi) 


If there exists an open sphere S(9^,p) with radius p >0 centered at 

2 

9 such that 7 ^ constant rank n 4-P at ?■! and for all 

0 CF pw) 

^S(9^,p)t then the parametrisation is locally identifiable from the 
transfer function at The matrix ^^^P*9) evaluated at the point 




I 


i 

i 

\ 



-^4 
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(l»i) is given by 




(i.») 


«nx»®'’'<S)-^<S)®W 2 2 


n xn 




nqxn' 


rnx» 


O 


rqxn‘ 


2 

n^xp 


G(9) 


nqxp 


H(0) 

/ 

D(«) 


rnxp 

rqxp 
(3.25) 


2 2 

which is an (n +nq + rn + rq) by (n +p) matrix. 

We give an example to illustrate the above theorem. Consider the 
system parametrized by 



G » 



I 


H - C 1 0 ], 

D = C 0 ] 

e • C«1 «2 


(3.26) 


then 




S3 




(I.«) 


» 1 
0 0 0 


» 4 

0 0 10 

0 0 0 

1 0, 0 0 


0 0 0 1 

1 0 0 

2 

0 0 0 e. 

m 

0, 0 0 0 

0 0 0 

1 

0 0 10^ 


1 2 
0 0, 0 0 

0 10 

2 


1 2 \ 



0 0 0 
0 0 0 Oj 
10 0 0 
0-100 
0 0 0 0 


0 0 0 
0 0 1 
0 0 0 
0 0 0 
0 0 0 


0 -1 0 0 0 0 

1 «„ 0 -110 0 

-Oj 0 -e^ 0 0 0 

0 -e^ 1 0 010 

0 0 0 0 0 0 

0 0 0 0 ^ 0 0 1 

-1 0 0 0 0 0 0 

0 -1 0 0 0 0 0 


0 0 0 


(3.27 


2 3 

which is of rank 2 + 3 >■ 7 for all hence the parametrization is 
globally Identifiable from the transfer function. 

We will extend the above theorem for the case that 0 includes the 

r^oi 

unknown initial state, i.e., J 1 , where j] is the unknown system 

, Li ] J , 

parameter vector contained in |,F,G,H,0} with a specified parametrization. 
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Then the aupnented parametrization Is a mapping from the parameter apace 
n to the space. Let «n(n+q+ r) + rq + n. We note 

that 


and 


-o"^^nxn‘ °nx(p-n)^- 
2"^®(p-n)xnJ ^(p-n)x(p-n)^ 


(3.28) 


500 


LetJ 

J 

following set of 


denote the true parameter, 
equations is sought 


The solution (P»0) of the 


PF(T1)P‘^ = F(t|q) 

PG(a) » G(t1q) 

H(3)p‘^ - H(2q) 

D(t]) - D(t]q) (3.29) 

The following theorem is established immediately. 

Theorem 3.5 

I > 

Let (^tFfG,H,D}(9):(l-*R be a given parametrization of a system 
Z and suppose CPiG,H,D} is minimal. Let 


fi*(p,e) 


^^0 

pf(3)p 

PG(T}) 

H(2)p” 

D(3) 
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If thtre exists sn open sphere S(Oq»p)cO with redlue p>0 centered 

it 9 

St Oq such that (P|9) has constant rank n -fp at P ■ end for 

all 9cS(9q»p) • then 0^ is locally identifiable (by definition 3.2). 

evaluated at (l^j^u*®) i® given by 




( 1 , 0 ) 


^^nxn®-0 K.. 2 


nxn 


j^O nxp 


(!])-F(ri)©I^„) (7^P(r,)) 

n xn •* n‘xp 


nqxn^ 


(-Hto)®t ) 

mxn 


o 


rqxn 


(!)))„„,„ 

a nqxp 

(7J(T])) 

s ^ rnxp 

( 7 ^ 5 ( 2 )) 


rqxp 
(3.30) 


which is an by (n^-f p) matrix. We note here that x^*Xq since is 

a column vector and hence VtMn * [l * 0 . . ] by (3.28). 

9-0 •* nxn- nx(p-n) ^ 

Comparing Definition 3.2 and 3.6, and Theorem 3.4 and 3.5, we see 
that Identifiability from the transfer function is equivalent to the 
zero-state parameter identifiability according to Definition 3.2 which is 
a more general definition. 
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3.4 Region of Leaet-Squere Identifiabllity and 
Tranafar-Funetion IdantlflablliCy 

In this section wa will discuss the relation between the least-square 
identifiabllity and the transfer- function identifiabllity. We consider 
a parametrized single -input, zero-state linear system Z with system 
matrices {f,G,H,D). Let the true parameter be 0^. Its measured inpulse 
response matrix is given by 



* i(0) ‘ 


' D(So) 


^(1) 


H<§o)G(§o) +D(9 q) 

Y ■ 

1 ( 2 ) 

e 

e 

m 

H(«o)F(§o)G(So> +»<So^G(9o) +D(9q) 

e 

• 


• 

i(M) 

» 4 


e 


which is an r(M-f 1) -component vector where r(M + 1) 2p. 


(3.31) 


The output sequence generated by the unknown parameter 9 fdien applied 
with the impulse input is given by 



■h(0;9)‘ 


D(9) 


h(l; 9) 


H(9)G(9) +D(9) 


h(2; 9) 

e e 
• 

m 

H(9)F(9)G(9) +H(9)G(9) + D(9) 

e e 
• 


e 

h(M; 9) 

» < 


• 

cJqH(9)F^(9)G(9))+D<9) 

s 


Let the identification criterion be 
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Jm(«) * 5) - y(k)]^ Ch(k; SI - y(k)] 

- CSh® - V*CJ!m(S) -V 

Then 

4„(«) « 1/2 J^(») - Clv;(«)f[j!^<S) 

which Is a p'component vector function, and 

4^(8) « + 

which is a (pxp) matrix function of 0. 

We note that 


W ■ ^ • 

^(So) - 0 . 

«<• 4^<»o> - Ch-ao/fCh^^j)] 

Recalling that the Markov parameter matrix is defined by 


D(§) 


M(S) • 


H(9)G(§) 

H(§)F(g)G(e) 


H(§)f”"^(9)G(§) 


By 


performing row reduction on 


2^^) , its easily seen that 


■ rank[h^(9)] for all ©eO 

■ rankCh'(0)] for all Ocfi. 

“M ““ 


( 3 . 32 ) 

( 3 . 33 ) 


( 3 . 34 ) 


( 3 . 35 ) 


rankCH(9) ] 
and rank[Ti'(§)] 


( 3 . 36 ) 


S8 


Moreover, from the metrlx theory we know thet 

renk[^(0)] ■ p If end only If renkOQi^(£))^(i!^(^)] *P* 
For brevity, let 


and Nj,(S) n [h,J(§)]^i:h^(9)]. 


(3.37) 

(3.38) 


Our purpose is to find a region S(§^,p) such that 8^ Is both least- 
square identifiable and identifiable from the transfer function in S(Oq,P). 
We first state the following Lemma. 

Lemma 3.6 

Let B and C be two (n x n) matrices. Let ||*|| be a norm on the space 
of (n X n) matrices . 

If (1) B is nonsingular, 

(li) ||B“^||sa, 

(iii) ||C- B|| £6, and 
(iv) of6 < 1 , 


0/6 


then C is nonsingular and Ijc'^lj 
Theorem 3.7 

Let E be a zero-state, single-input linear system parametrized by 
the unknown parameter 8. Let the true parameter be 8^. Let ||*|| be a 
norm on the respective spaces. If 

(1) ^(§q) nonsingular, 

(2) there exists an open sphere S(8^,p) with a radius p>0 centered 
at 8^ and a set of positive numbers (\,y,m>) such that 
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(li) llCi^(§)3\<S)||^Y for all ««S(8o.P). 

(Iti) llNjJ(§)|! « for all 5cS(§^.p) . 


(iv) X(hp+y)< 1» 

then Oq Is both least-aquare identifiable and identifiable from the 
transfer function in S(§^,p). 

Proof: By 2(iii) we have 


\\\(S) i*S<§Q,p) 

Moreover, ||N^ ^(§q)||^X by 2(i). Applying Leonia 3.6 to this situation, 

we have or«\, 6"p|x, and Xpp,<l by 2(iv). Thus N (§) is nonsingular and 

M 

for .11 5,S(«^.p) 

Since 3 and %(§) has constant rank p in S(0^ >p)> 

therefore h|!((§) has constant rank p in S(0^,p) and M(§) has constant rank 

p in S(§Q,p), hence 9^ is locally identifiable from the transfer function 

by Theorem 3.3. We now show that J^(9) is nonsingular for all 9cS(9 ,p) 

and J'(9) is locally injective on S(9 ,p). 

M 0 

By <3.34), 

4 ^( 8 ) 

therefore 

for all 9cS(9 ,p) by 2(ii). Applying Lemma 3.6 again, we have a - — ^ , 

“ w I " X|ip 






6 "Y* cr6 - < 1. Thus i* nontingular, i.e., 
constant rank p for all ^S(0^ip). This iaqillea that 9^ is 
least-square identifiable by Theorem 3.2. 

A computation procedure for finding explicitly S(9^,p) 
presented in the next chapter. 


has 

locally 
will be 


61 


4. REGIONS OF PARAMETER IDENTIFIABILITY FOR 
DETERMINISTIC LINEAR DYNAMIC SYSTEMS 

In this ehsptar we will study the identification algorithm exten- 
slvely. We will first study Pereyra's [133 theory on the modified Newton 
methodt which we shall call the Gauss-Newton method, for solving nonlinear 
least square problems. Then we will present the identification algorithm 
proposed by Herget [7] for finding the regions of parameter identifi- 
ability. 


4.1 Nonlinear Least-Square Problems and 
the Gauss-Newton Method 

In this section we will study the sufficient conditions for the con- 
vergence of the Newton iteration sequence and the Gsuss-Newton iteration 
sequence. 

Definition 4.1 

A real- valued function ||*|| defined on the R** space is called a 


norm if 



(i) 

y|io 

for all mr", 

(ii) 

y-o 

if and only if x ■ 0, 

(til) 


** Ibsll Ibdl «11 and 

(iv) 

llo*!l " 1 

|cr||lx|{ for all oeR end all xcr". 


Definition 4.2 

A mapping is said to be Frechet-differentiable at C, 

where 9 is an interior point of Q, if there is an (m x p) linear operator 
A such that 

lla U/|l!fl|)|lt<«+ k) - ♦«) - *k|l •« 

||k||-0 


for all k such that 9-f kcCl. A la danotad by and la callad tha F- 

dartvatlva of if at 9. 

By a laaat aquare problam, wa maan glvan a nonllnaar tranafonaatlon 

h:ncR^-'R"‘ batwaan tha tat QCrI’ and tha R.'" tpaea (psm in ganaral), 

and the tequence of obaarvationt YaR*'^, find a paraioatar O^aQ auch that 

the Euclidean norm of h(9) * Y la mlnlmlaad at 0^, l.e., we want to find 

2 

the atatlonary polnta of the criterion ||h($) - Y|| . 

For brevity, let f(§) *h(S) “X* £(9) la at laaat twice Freehat- 

differentiable on 0, then we can differentiate the criterion to find the 
minimal point of the criterion. Let 

J(«) - Ch<§) -if Ch(S) - Y] <g) 

which la the aquare of the Euclidean norm of h<9) - Y. Then 

^(9) m 1/2 J'(9) -Cf'<9)f f(§) (4.1) 

where I '(9) la an (mxp) matrix function of 9 and ^(9) la a p«component 
vector function of 9. 

To find the stationery points of J(9), we let 

^(9)-0 (4.2) 

Many questions arise. (1) Does there exist some 9^«0 such that J(9^) *0? 
(2) If 9 q exists. Is 9 q unique locally or globally? (3) If 9^ Is unique 
locally in some region, can we find explicitly the region? (4) If 9^ 
exists, how do we construct an Iteration sequence idilch will converge 
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To antwor thoM quootlont, «• proeood •• follows. Tht aemdard 
Nowton Itoratlon aaquanea for aolving 6ha aquation 0(6) "0 ia dafinad by 




(*. 3 ) 


whara 


<S> I' iS>* Ci' <8) 3^1 (S > . 


(4.4) 


and j"(6) ia an mxpxp bilinaar oparator dafinad by 

^ i-1,— ,m 

k-1, — ,p 


(4.5) 


The uanipulacion of the bilinaar operator follows that given in Sec. 3.1. 

The Newton* Kantorovich Theoren provides sufficient conditions for the 
convergence of the iteration saquence (4.3) and the uniqueness of the 
solution for (4.2) in a region. 

Theoreis 4.1 [Newton- Kantorovich] [l2] 

Assume that is F-diffarentiable on a convex set 

and that 

II* '(Sl) -4'<Sj>ll ‘yIISj - ijll tor .11 

Suppose that there exists an such that jl2'(SQ)||^9 *nd a«0yr)X]/2 

where |lCi '(i^)]‘^i(iQ)|| <r,. 


Pl-(M"kl- (l-2o)’l. P 2 - (6v *[l+ (1- 2o)’], 


and assume that where S(§^iPj) Is the closure of the sphere 


6A 

8(S^»p^) with radius eantsrsd se Than thn lesratlon snqusnes 

mm ^ 

is vall'dsflnad, rsmslns In S(fiQ»pp *nd eonvsrgss to a aelutlon 9^ of 
^(•) >0 which is unique in S(0^(P2) Moraovart tha arror astlaaea 

holds. 

By (4.4), wc sea that by using Nawton's mathod, wa oust cslculsta 
J"(Q) which is a billnaar oparstor in aseh itarstlon. This is a conpu- 
tationsl inebnvanlanea. Tarayra [13] davalopad a nodlflsd Nawton msthod 
which wa shall call tha Gauss*Mawton iMthod. Tha ■odlfleation la that 
^'(8) is raplacad by its first ordar approxlsMtlon tha 

flwdifiad itaration saqusnea is daflnad by 

4(i„). n-0,1.— (4.6) 

Dua to the approxinatlon on^'(§), another set of sufficient conditions 
for the convergence of (4.6) and the uniqueness of the solc'.lon of (4.2) 
was established by Pereyra [l3]. 

Theorem 4.2 

Assume is y-differentiable. For brevity, let N(§) m 

Ci'<«)]^i(§). If 

(1) N(8q) is nonsingular, 

(2) there exists a sphere S(8q,p)cO of radius p cantered at ^ such 
that 
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( 1 ) 

(11) ||li''(8)]^(fi) - U‘’5o)]^(i^j)(I’'V for all 

(iiO l|Ci'(^)fi(So)|l*Y 

(iv) !|h'( 8)||«,* for oil Ji8(^.p) 


(V) 

(vl) 


\(uP + y) < 1/2, 
Vf^Pp tfhoro Ti ■ 


X||»(^)|l 

1 * iVciAp-t-y) 


Than tho Itoratlon soquonco deflnod by (4.6) eonvorgtt Co tho unlquo 
solution 9^ of 2(2) -0 In tho sphoro S(9^,ri). Moroovor, cho error 
•tclmco It glvon by 



whoro k ■ 2X (p,p •»> y) • 

Tho above theorem provides a convergence region centered at the 

Initial iteration point such Chat the solution of (4.2) Is unique In Chat 

region. If ue have the knowledge of the solution of (4.2), then we are 

able to find a region centered at Che solution 9^ such that 9. Is the 

-0 -0 

unique solutloTi of (4.2) In that region by modifying the above theorem. 
This will be studied In the next section. 


4.2 Regions of Parameter Identlf lability 
After introducing the theory and algorithm for solving the nonlinear 
least'squarc problems, we are now going to apply it for Che control system 
parameter Identification problem. Ue shall first introduce Che I -norm 

(I) 

which will be employed by the identification algorithm developed In this 


section 
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The well-known class of norms on R** space is the t^-norm defined by 

Hp- . 1<P<- 

When p«2, the t^~nom is usually called the Euclidean norm. 

The limiting case of the t^-norm is the 1.^-norm defined by 

• Max |x. 
l«i«n 

We can thus now induce a norm on the space of linear operators from 
R to-R*". Ue denote this space by L(R^,r!°). Given any two norms ||*|| and 
11*11' on R® and R® respectively, and any AcL(R®,R®), the norm of A with 
respect to ||*|| and ||*||' is defined by 

I|A|| - «»p ' 

tell - 1 

Such a matrix norm satisfies the properties: 

(i) IIaII^O for all AcL(r“,r“), 

(ii) |jA||«0 only if A*0, 

(iii) ||aA|| ■ |of| ||a|| for all areL(R°,R**) , 

(Iv) ||A+ B|| ^ ||a|| + ||b|| for all A,BeL(R",R*°) . 

The 't>^-norms are the most useful in numerical analysis 

work. For the system identification problem concerned in this section, 
we will employ the t -norm. We denote the t -norm of a matrix A by ||a|| . 

00 CO CO 

The following theorem provides the explicit expression of ||a||^ [12]. 
Theorem 4.3 

Let AeL(R®,R®) where both R® and R® are normed by the -L^-norm. 

Denote A * 
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Then 



Max 
1 < 1 


n 




Proof: For any xeR*', 


(4.7) 



■ Max I (.Ax) I 
ISiSm *■ 


■ Max 
1 s; 1 fim 




$ Max 
1 



S Max .LUnl‘( Max jx J) 

isjin ^ 




(4.8) 


l^i^m 

It sufficies to show that there exists an x cr" such that the equality 
is attained in (4.8). Let k be the index such that 


Max 


Define x by 


* fV 
J I 1 




■ ’kj“” 


j ~ 1»““**** 


Then 


II* IL*i* 
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1 S i :Sm ^ 


- Max I 8 a x* I 
l^l^m J»1 J 


If i#k, then 

n * n 


‘lll'*ijlllsT*o’‘*l 
- llj*ijl 
* 


If 1 « k, then 


'i?i‘*j’‘jl 'iSiKil 


Thus 11^ 
attained. 


^ n 

- L* equality in (4.8) is 


We can also induce an <.^-nomi on an (m x p x p) bilinear operator B 
defined by 


I|bIL 


sup 11^1 


lx - 1 


We now consider the problem of Identifying the vector of unknown 

parameters r^, and the unknown initial state w(0) of a parametrized 

deterministic system whose state at time k, is the vector, w(k) , where 

I 

k-0,1,2— . Let x(k) -I I , which we shall call the augmented state 
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vector, and let x(k) be a p-component vector. Then 9 ■ x(0) 



to be identified. We assume that the true parameter 9^ ia an interior 
point of a known compact subset ClCR^, and the system is stable for all 
admissible inputs and SeQ. We note here that after the augmentation, 
the identification problem of the original system is equivalent to the 
initial-state observation problem of the augmented system. We assume 
that the fimctlon ^(x,k) Is known as a function of x and k and that 


x(k+ 1) -£(x(k) ,k) 


(4.9) 


We also assume that the Inputs to the system are known implicitly 
in ^(x,k) . 

Furthermore, we assume that observations of the state can be written 
in the form 


^(k) * Qc(k) 


where c is a known (r x p) constant matrix. 
Let 


2 ( 0 ) 

^( 1 ) 


y(M) 


(4.10) 


(4.11) 


be the m-component vector of observations, where m- r(M+ 1) ^p. Let 
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Cjc(O) 

Cx(l) 


(4.12) 


[ Cx(M) j 

where x(k) is the solution to (4.9) when x(0) > 9. Moreover, let 

4<S)i !!„<«)- X„ 

We assume that we have obtained a solution, say 9^, such that 




(4.14) 


l.e., ^ Is the parameter value which when applied with the Input sequence 

from a given set of Input-output sequence will generate a set of output 

data which will match the given output sequence. In order that 9^ Is 

Identifiable, 9 q must be an Interior point of an open sphere S(9^,p) cO 

with radius p > 0 centered at 9 such that ^ Is the unique solution to 

-0 ”0 

the equation 


f^(9)-0 (4.15) 

In S(9 ,p). We are now going to establish the sufficient conditions for 
0 

the existence of S(9 q,p). Let the Identification criterion be 

J„(«) i 

We see Inanedlately that >1^(§) ”0 If and only If J|^(§) *0* Since Jj^(9) Is 
nonnegative for all 9, therefore Its minimal value Is zero. If we have 
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the knowledge of 0^, then ^ minimal point of 

. Let 

- 1/2 J^(0) -C^(S)f4(g) » (4.17) 

then 

^m(«) - Cfj^(«) ]’^^(«) + Ci^(S) ]\(g) (4.18) 

Let 

N„(«) i CiM(S)f^(S) (4.19) 

Note that 

^j,(Sq)-0. (4.20) 

and 

( 4 . 21 ) 

Thua if there exists an open sphere S(0^,p) such that ^(§) is non- 
singular for all 0eS(§^,p) , then j^(0) is locally injective on S(0^,p) 
and thus 0^ is locally identifiable by Theorem 3.2. The following theorem, 
established by Herget [7], provides sufficient conditions for the existence 
of such S(0^,p). 

Theorem 4.4 

Let R and R be the vector spaces of p and m- tuples respectively 
over R. Let ||*|| denotes any norm on the respective spaces. Let ^(§) he 
an m-vector function of OcR^ which is twice F-differentiable on QCR^. 
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If (1) there is ^cQ such thet ^($q) 

(2) N^(§q) noneingular, 

(3) there Is a sphere with radius p >0 centered at 9^ and a 

set of positive numbers that 

(ID ||[^(«)l^(»)|| *Y tor all S«S(Sj.p) , 

(ill) |lN^(9)||Si* for all 9eS(9Q,p), 

(iv) l|i^(§)|lsTi, for all 9 sS(9q,P) , 

(v) X.iiSp/4, 

(Vi) \(2 hp+y) Sl/2, 

then 9^ is locally identifiable in S(9 q,p). Moreover, for any ^cS(§Q,p/2) , 
the iteration sequence 


converges to the unique solution 9^ of f (9) »0 in S(9 ,p/2). 

Proof: We shall first prove that N^(§) nonsingular and ||N^ ^(9)||:SX. 

for all 9cS(0^,p). By 3(111), we have 

||n„(«) - N„(9^)|| «up for all 9cS(9^,p) 


Applying Lemaa 3.6 to this situation, we have cir*X/2, 6 ■ pip and ■ 
Xp,p/2sl/8. Therefore N^(9) is nonsingular and |jNjj ^(9)||^- 

l+(^-Xw>^^ fotall 2 

A result due to Bartle in Herget [7] Is: If |!ijJ(§) *9 fot 

all 9cS(9^,p), then |jfi„(9^) -^^(O^) -^^(So) (g^ - 9^)11 ^ e||9j -i^H for all 
9^,92«S(9 q,p) . In our case, *Njj( 9^) by (4.21). Thus by 3(il) and 
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(ill) 

lb^(S) -j;<So)|| - IIHh® - + CiJ<S)]\<S)|l 

< (ip for all J*S(9q,P) 

Therefore we have 

ll*H<Sl -4 m»2’ -4.;<2o><Si ■«2>II * (M+V)||«i - Sjll 

for all . 

Also, look at 


Thus 






Let 9 eS(§Q,p/2) , and 


want to show that ^cS(§Q,p/2) for all n. We prove it by induction, 

- 1 , 


We 


Thus llfij-9 |lsXr)Sp/4. Hence 0j^eS(9^,p/2) and 9 «S(9 q,p). Now assume 
9 ^sS(5q»p/ 2) forv«l,— -,n. Then we have 




7 * 


n»» %,<•,) -4 h<S^,) 

Th.r.tot. t|4H(»,)||*W+v)||S,,-i^,ll + l‘P||8^-5^ill - kllSv-Vill 
where k>2g,p-fY* 

which implies 
Look at 

^ ^ ^ 1 ^ ^ 

Therefore 

^ ^ fl ^ ^ 

e , - 0^ • I CS , - ® 3 

- trt-l “0 v »0 •'>♦'1 “V 


Note that 
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Thus 


n M 


ll2„i-Soll‘A»Svrt-Svll 

n y <»* 

‘v£o»« IlsrSoll 

._!_li; .;i| 

‘ i 

‘ok ■I-''* 

which Implies 0 ^j^cS(0q,p/ 2) , thus the induction proof Is cosq>leted. 

Next, we shall show that (O } . is a Cauchy sequence. Let ni>n and 

n n*o 

m * n J , then 

^ ^ ^ n I 1 ^ 0^ 

e-e.e -o.j; [® -®], 

“B ~n “n+J “n v*n ”\h-1 


Thus 


1^ .-0 II s z: III -e II 

iH-J “n v»n '^NH'l V 

n+J-1 y ^ ^ 

s Z (\k) |1§, -ej 

v.n ^ O' 


(xk)"c^i:‘«k)'']||» II 

v»0 1 0" 


s (Xk)”C Z (Mt)'']|li -0 II 

v-0 ' 1 “0 


iiii -iofl 
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(f) 

1 • \k V 
^ 1/2 


- (Xk)"(|) 


Since \k<l/2t therefore 11m (\k)*^(£) >0 end hence 

n-*cB ^ 



is e Ceuchy 


sequence. 

Since R is s coaq>lete nomed space, CSq}q.q converges to some point 
9^«S(6^,p). We must show that On the contrary suppose 

then we can write 


- 1 , 


io-Sl-"M «oW<So-V 


■Ince - 0 

Thus 

IlSj * Sjll * ^ <w +v + upMISj - *jll 
‘ 1/2 IlSg -1,11 

which is a contradiction. Thus 9 «9 and the proof is completed. 

"0 “1 

Remarks : 

(1) S(9 ,p) is the region of parameter identifiability, i.e., 9 is 

0 ”0 

the unique solution for f (9) "0 in S(9 ,p), but the convergence 
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of dio Gaut«-N«wcon Itoratlon Mqu«ne« Is assursd In 8(|g^,o/2). 
This is sn sddsd fsseurs. Howsvsr, our priswry intsrsst is tbs 
uniqusnsss of ths solution of * 0 1 b tons rsgion in 0. Ws 

hsvs sssunsd us slrssdy knou 

<2) If 1 VS sre concsrnsd shout ths uniqusnsss of ths solution only, 
thsn conditions 3(i), (ii)i (iii) of Thsorsn 4.4 and <2 

suffics for ths sxistsncs of S(9^,p). To provs this, ws supposs 
that thsrs is s •jtS(8Q,p) such that l**sn 

Thus we can writs 

Thus 

•=llSo--Si« 

which is a contradlcCion* The region implied by these set of sufficient 
conditions will generally be of larger radius since X(up*t”y) ^1/2 is more 
conservative than \(up -t-y) 2. 

(3) Theorem 3.2 is in^lied by Theorem 4.4. To prove this, we first 

recall that N (§) is nonsingular and ||n *^(9)f|<\ for all 
M M 

5«5(Jj|,p) . .nd ■ *^(8) ♦ tC® 
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I 


for all jcS(S^,p). Apply Lonm 3.6 again, m hava a«X, 6 -y 
and Xya 1/2 <1. Thua ^(§) !• nonaingular for all ^8(S^,p) 
and ^(8) ia locally InJocClva on 8(§^,p). 

Ha now darlva a aac of raeuralva focantlaa for eonputacionally veri- 
fying condition 3(1) to 3(vl) of Thoorom A.4 for the caaa of dynaadc 
ayatana. 

Recalling that 


Thua 





’ Cx(0/ 


,2(0) ‘ 

• 

<?x(l) 

0 

a 

m 

2(1) 

a 

a 


• 

Cx(M) 

- 


a 

2(H) 

u 


C|x(0) 


2(0) 

C2(x(0),0) 

a 

a 


2(1) 

a 

a 

a 

Ci(x(M- 1),M- 1) 

» m 


. s 
>1 

1 



C 

Cj'(x(0),0) 


(4.22) 



3 


I 

i 

i 

i 


Cg'(x(M- 1) ,M- 1) — i'(5(0),0) 


>9 


cr^(S) 

cr^(S) 


[cr„(8) 


(4.23) 


whor* T^(S) -i'(x(M- l),M- 1) (8), k-l.— ,M 

and r^(S)*l ■ (pxp) idantity matrix, 

O'* pxp 



cr^(8) 

CT'(g) 


CTjJCg) 


(4.24) 

(4.25) 


(4.26) 


where rj^(0) -Ci"(x(k- l),k- 1) r^.^(fi)3* 

+ j'(x(k-l).k-l) k-l,— ,M (4.27 

and 

r^(«)-0 (4.28) 

To confute the bounds on the norms given in the hypothesis of Theorem 4.4, 
we choose the t^-norm since the procedure is relatively straightforward 
If we employ the interval arithsMtlc [H]. 

Let 

(set of all finite closed intervals [a,b]:a,btR,s £b} 

The interval l*[a,a] is called a degenerate interval. The interval 
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arithmetic operations are defined by 

{x*y:xtI»ycJ} for all I, Jca9 (4.29) 

where the symbol "*'* indicates one of the arithmetic operations 
and /« except that I/J Is not defined if OeJ. 

For example, 

[a,b] + [c,d] « [a + c,b+ d] 

[a,b]*[c,d] « [mln(ac,ad,bc,bd) , max(ac,ad,be,bd)] 

[a»b]/[c,d] «[a,b]*[l/d,l/c], provided 0^[c,d]. 

If the real number § is to belong to the closed interval on the 

real numbers, § denote this interval by 

id R 

[ 5 ] S [ 1 ^. 5 ^] for brevity. 

If ^(x) is a continuous, real- valued function of xsR, Chen Che interval 
function [i|r([x])] is defined by 

[♦([*])] 5 {y:y = ♦(*).«[*]} 

An interval function will be called a raticnal function if it is 
defined and can be expressed as a rational interval arithmetic expression 
in the interval variable and a finite set of constant coefficient intervals. 
For simplicity, we shall assume that all of our functions of are 
rational functions so that interval arithmetic suffices to evaluate their 
norms. It is always true that the true interval function is a proper 
subset of Che computed interval, i.e.. 
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U ♦([x.x]) s [♦([x])] c Ii|t([x]) 
xc[x] ^ 

where It([x]) Is the computed interval obtained by replacing x by [x] and 
evaluating ijr by Interval arithmetic Instead of ordinary arithmetic. The 
following Theorem in [ll] proves the above statement. 

Theorem 4.5 

Let l*[a,b]c/vP and 

/V?I • 

Let t continuous on^^}, and d be a metric on j|[ such 

that ({_»d) is a complete metric space. For any rational function i|rc(j[,,d) 
and arbitrary Je^^, 

U t([x,x]) c IMJ) 

xs[x] * 

Proof: From (4.29), it follows that If I,J,k,Lc^, ick and JCL, then 

I * J C k*L 

provided in the case of division that OdL. This property of Interval 
arithmetic is called "monotonlc inclusion". Hence the result is obvious 
from the monotonlc inclusion property and the definition of the rational 
Interval function. Since a finite number of these operations is involved 
and since for every xcJ [x,x]cj, [x,x]s^^, then KCx.x]) cl i|r(J). To 
prove that equality need not be attained, it suffices to give an example. 
Let *(x) *x^, J- [-1/2,1] 

Then It|r(J) = « [-1/2,1], 
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but Ct([x3)] -[0,1], 
thus [-1/2,1] t [0,1]. 

We can now apply the interval arithmetic to the evaluations of norms 
required in Theorem 4.4. 

The closed sphere S(0^,p) is the vector interval given by 

V ■Ji'-V ■ “•V* 

which is the Cartesian product of closed intervals. We consider condition 
3(iv) of Theorem 4.4 first. We wish to have a bound on 


Let 


sup 

i«S(§o,p) 



sup Max |0„^(§)1. 

0«S(§Q,p) isiSp 


► 



then 


The computation of I^^([0 q]) is generally much easier than that of the 
[tf ([^])], moreover, by Theorem 4.5, 


l^,S<tSo])n3 


(4.31) 


Hence we will compute the right hand side of (4.31) as the bound of 
the norm of ^(9) . 

For condition 3(ii) of Theorem 4.4, we let 



1 

I 

1 

k 

\ 

■i 

1 


i 
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A(S> i 


(4.32) 


for brevity. 
Then 


sup ||A<e)||« 

i*S(SQ,p) 


Max [.l,Max{|l,a 

1 sisp J ^ 


IJ 


(tioDl. IVij<C2o^>l» 

(4.33) 


For condition 3(111) of Theorem 4.4, we let 



B(9)-[i>ijk(s)]iN,;(») 

(4.34) 

where 


>i^(s) - i^(s) fi;;®) + (1^(9) ]\(«»* 

(4.35) 

Then 


sup 11 8(0) 11= sup [ sup j|B(§)^||] 

i«s<So,p) »«s(e^,p) y-i 

(4.36) 

We note that |[£j| = 1 if and only if 5^^ *±1 for some 1 = 1,— 

— ,p. Hence 


we Introduce the following notation. For l,n«l,—« ,p, let 

Clil =C-1,+1] if tj^n 

and * C+1,+1] 

For i*l,--~,p and n»p+l, — -,2p, let 

[5^]^ = C-l.+l] if i - p 

[§.] =[-1.-1] ifi = n-p 

1 n 

Then we have 


sup ||b^)||s 




(4.37) 
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We sunaarize this algorithm by the flow graph In Fig. 1. 

4.3 An Example of Computing the Region 
of Parameter Identlflablllty 

If a system Is linear, time- Invariant, and Its parametrlzatlon Is 
known, then Its augmented system has the quadratic- In- the-state bilinear 
representation. 


x(k+ 1) - [F + Dx(k)]x(k) + [g+ to(k) }u(k) 

i(k)«Cx(k) , (4.38) 

i • 6 a g 

l(x,k) -[F+Iht]x+[G+to]u(k), (4.39) 

where u(k) Is a q-vector, F Is a (pxp) matrix, G Is a (pxq) matrix, D 
Is a (pxpxp) bilinear operator, and E Is a (pxqxp) bilinear operator. 
Then 

1 (x.k) -F+ (D + D )x+E u(k), (4.40) 

and 

£"0S.k) -D + D* (4.41) 


We now give an example to Illustrate the algorithm of Theorem 4.4. 
Consider the parametrized system which has already been given In Sec. 2.3. 


* « 
w^(k+l) 

m 

\ '■ 


t 

< 

1 

+ 

• 4 

0 

w (k+1) 
2 


0 


CM 

> 


’’3 

• 


> 


L J 


• 4 


i(k)-C 1 0 ] 


Wj(k) 


u(k) 


k-0,1,2,— 


(4.42) 
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The block diagram of thla ayatem la depicted In Fig. 2. Ue note that 

by reveralng the poaltlona of and -X— , and by tranaformlng the 

a-T)2 

Initial condltlona by a almllarlty tranaformatlon, the reaultlng equivalent 
ayatem will generate the aame output aequence. The equivalent tranafor* 
matlon matrix la given by 


P 


1 0 

TI1-H2 1 


which haa already been obtained in Sec. 2.3. The parameter 8 b[wj^( 0), 
^2* ^3^^ Identified. Letting x(k) "[w^Ck), W 2 (k), 

‘> 1 l> ^ 2 * ^ 3 ^^* system can be written In the quadratlc-ln-the>state 

bilinear system form as given In (4.38) if we let 


F 


0 1 0 0 0 
0 0 0 0 0 
0 0 10 0 
0 0 0 1 0 
0 0 0 0 1 


0 

0 


G • 


0 


0 



f 
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00100 00000 00000 00000 00000 
00000 00010 00000 00000 00000 
00000 00000 00000 00000 00000 
00000 00000 00000 00000 00000 
00000 00000 00000 00000 00000 


rad 


0 0 0 0 0 
0 0 0 0 1 
0 0 0 0 0 
0 0 0 0 0 
0 0 0 0 0 


C - [1 0 0 0 0] 


We have used the notation in [i4] for representing the bilinear form 
D, and E degenerated to an ordinary matrix since u(k) is a scalar. 

X 

Let 0«[wj^(0), W2(0), TI2. Tij] . It can be easily seen tliat 

0 «[w^(0), W2(0)+ + '^ 2 * ^1* ^3^ ® ^ which is 

unresolvable from 0 by the equivalent transformation. Hence the solution 

to the equation * 0 is not unique in and only local identifiability 

can be imposed on the parameters of systems (4.42). We applied the 

algorithm of Theorem 4.4 to this example with 0^>[O.5. 0.1, 0.3, 0.7, 

1.0] . We note that 0^»[0.5, -0.1, 0.7, 0.3, l.O] will give exactly the 

same set of j[(k) sequence for any input sequence u(k) and hence is 

unresolvable from 0^. However, the distance from 0^ to 0, is 0.4 by using 

the t -norm. Therefore 0. and 0. are each locally identifiable in the 
00 “*0 “1 






87 


spheres S(Oq,p) and S(8^,p) respectively Where p<0.4. By applying 
Theorem 4.4 and the algorithm developed In the above section, we are able 
to find two spheres centered at 8^ and 9^^ respectively such that 9^ and 
9^ are locally Identifiable In those two spheres respectively. To 
demonstrate this situation, we use an Input sequence u(k) ■ 100 sin (kn/4) , 
for k>0,l,">, 19, we found that with twenty observation, the sphere 
centered at 9. is of radius p>0.21 and 

X - 0.1314x10”^ 

71 - 0.3646x 10^ 

Y - 0.2284 X 10® 

H • 0.2124 X 10^ 

\Ti » 0.0479 <0.0525 - p/4 
X(2p,p+Y) ** 0.312 <0.5 . 


Thus the hypotheses of Theorem 4.4 are satisfied, and we conclude 

chat 9 Is locally Identifiable In S(9^, 0.21) and the Gauss-Newton 
-0 ' '“O 

sequence will converge to 9^ starting from any point In the sphere 
S(9^, 0.105). We also tested the conditions of Theorem 4.4 about Che 
point 9^ and found that p>0.31. Hence 9^ Is locally identifiable In 
S(9^, 0.31) and the Gauss-Newton sequence will converge to 9^^ starting 
from any point in the sphere S (9^ , 0. 155) . 

Since Theorem 4.4 gives sufficient conditions for convergence, and 
because of the upper bounding implied by the use of interval analysis, the 
question of whether these results arc overly conservative naturally arises. 






Is of reasonable size in view of the distance from 9^ to 9^^, i.e., the 
theoretical radius of the region of identifiabilicy. 


The conq>uter program for testing the conditions in Theorem 4.4 for 


the above specific example is listed in the Appendix. 





Figure 1. Flow Graph for Computing Regions of Parameter Identif lability 




P 

r 


O 


INITIALIZE: 

I 

BY 

EQ. 

(4.30) 



BY 

BQ. 

(4.25) 


^^0 

BY 

EQ. 

(4.28) 



FOR k - 1, 

••• t 

M 


COMPUTE: 

COHPUTE: 

I x(k) 

BY 

BQ. 

(4.9) 



BY 

EQ. 

(4.12) 


'A. 

BY 

BQ. 

(4.13) 



BY 

BQ. 

(4.24) 



BY 

BQ. 

(4.27) 


COHPUTE: X - 2||N^“ ^ (g^) (| 


LET: 


P "Po 

ITERATION - I 


Figure 1. Continued 



COHPUTE: 


BY 

BQ. 

(4.23) 



BY 

EQ. 

(4.26) 


I N' 
M 

BY 

BQ. 

(4.35) 



BY 

EQ. 

(4.17) 


I A 

BY 

BQ. 

(4.32) 


I B 

BY 

BQ. 

(4.34) 

Y 

RHS of 

BQ 

I. 

(4.33) 

V- 

RHS of 

BQ 

1. 

(4.37) 

r\ 

BBS of 

BQ 

|. 

(4.31) 




i 
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Figure 1. Continued 
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5. LOCAL PARAMBTBR IDENTIFIABILITY AMD 
LOCAL CML PARAMETER IDEMTIFUBILITT OF DYNAMIC 
SYSTEMS WITH MBASOREMENT NOISE 

In this chapter we will study the parameter Identification problem 
of general dlscrete^tine, snil tip le.input/multiple -output dynamic systems 
with measurement noise. Specifically, we will establish the concept of 
local parameter identlf lability and provide a computation procedure for 
finding the explicit regions of parameter Identiflabillty. Moreover, we 
will show that the constrained maximum likelihood estimation sequence 
converges to the locally identlf labile parameters with probability one. 

5.1 Local Parameter Identiflabillty and Local 
CML Parameter Identiflabillty 

Let ^ be a sequence of random vectors, which we shall call the 

observations, with joint probability density function — »z ; 9), 

u "Ti 

p p 

M« 0,1,2,-—, parsmeterlzed by the unknown parameter OeOcR*^. R is the 
space of real p- tuples with an arbitrary norm denoted by ||*||, and Q is a 
compact subset of We let Z * (z , z ,— ,z ), and we assume that the 

“n 0 ”1 

true parameter 9 lies in the Interior of Q. Furthermore, we asstmie that 
”0 

p(Z ;9) Is continuous with respect to ^Cl for Z almost everywhere, l.e., 
for e >0 and 9 c 0 , there exists a 6(e) >0 such that for all O'eQ with 
H §- 0'||<6 we have |p(Zjj;0) -p(Z^;0')| <e for Z^ almost everywhere. 
Following Tse and Anton [19], we make the following definitions. 
Definition 5.1 

Two parameters 9, 9, *^9 , are said to be unresolvable if the 

- 1-2 “1 ”2 

equality 

^^UCEDING PACE BLANK NJl rlLMLD 
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p(z )"p(2*52^ 

holds with probAbtllty om with rsspeet to and J 2 except a 

finite number of integers n>0, l.e., for all except a finite number of 
integers n>0« (3.1) holds with respect to the measure P(S^;9^)d^ as 
well as P(Zj,;e 2 )<* 2 „‘ 

Definition 5.2 

A parameter 9 cQ is said to be locally identifiable if there exists 
”0 

an open sphere S(Q^»p) cQ with radius p >0 centered at 9^ such that there 

is no other 9sS(9^,p), 9i^9 , which is unresolvable from 9^. 

- 0 0 

The problem of constrained maximum likelihood (CML) estisiation is 
as follows: find 4 sO, an estimate of the true parameter 9 , where 4 

is such that 


• Maxp(Z^;9), M-0,1,2,— 


(5.2) 


Since 0 is closed and bounded and P^;9) is continuous on (1 for 

Z almost everywhere, a solution to (5.2) exists. However, if 9. and 9 
“M * ^ 

are unresolvable in Q, then they cannot be Identified by the CML esti- 
mation method constrained to 0. Therefore the following definition is 
established. 

Definition 5.3 

A parameter 9^*^ locally CML identifiable if there 

exists an open sphere S(9^,p) with radius p >0 centered at 9^ such that 
the sequence ^ converges to 9 q with probability one, where 

IS constructed by 
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(5.3) 


5p8(«^,.p) 


S(S^>p) dmotet th« closur* of 8($^»p). U« will call S(S^>p) given above 
the region of paraneter Identlflablllty. 

We now eonalder the problem of Identlfyli^ the vector o£ unknown 
parameters, JQ, of a system whoae state at time k Is the vector w(k) %ihere 


k« 0,1,2,— . Let x(k) ■ 


w(k)‘ 

2 


and assuse x(k) is a p-oomponent vector 


which we shall call the augpwnted state vector. Let the initial state 


w(0) also be unknown, then 6>x(0) 


‘w(0)l 

U" 


the parameter vr^ctor to be 


identified. The identification problem is then equivalent to the initial- 
state observation problem of the augmented system. We asstme that the 
function ^(x,k) is known as a function of x and k and that 


x(k+ 1) *^(x(k),k) 


k-0,l,2,- 


(5.4) 


Observations of the state are taken which we will assume can be written 
in the form 


i(k) “Cx(k) 
s(k) -^(k) + v(k) 


k • 0, 1,2,-— . 


(5.5) 


where C is a known (r x p) matrix and v(k) is a Gaussian-white noise vector 

2 

with r components which has zero mean and covariane matrix Q - c l , 

rxr 

2 

c < 0 D. The observation sequence of system (5.4) and (5.5) is 


C*'^(0),z’’(l),— ,/(M)f 


and the Joint probability density function is given by 
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p(a^;S) )’^Q*^(i(k) -C^(k))l(5.6) 

^•re 2 <k) is the solution to (5*4) when ^(0) "9, end N**0,l,2,— . 

We essuBM that the structure of jt(x»k) Is such that the following 
assunptlons hold. 

(1) The Inputs to the system are known Implicitly In j^,k). 

(2) For every admissible Input sequence and all WsO» the states x(k) 
snd the deterministic part of the observations, w(k), genersted 
by 9 when applied with the input sequence ere bounded. 

(3) jg(x,k) is at least twice continuously differentiable with 
respect to 8, and hence so Is p(2^;8) . 

5.2 Regions of Parameter Identif lability 
Let the assuBq>tlon8 given In Section 5.1 hold, and lot us define 
. M « 

\(S> - ^ ^E^C«(k) -Cx(k)]‘Ca(k) - 05 (k)] 

for M- 0,1, 2, — . (5.7) 


Then the CML estimation method in Eq. (2) is equivalent to finding 
Min 1^(8) - . M-0,1,2, — . 

Following Che notation in Chapter 4, we let 


V)- 


Q|(0) 


ttcCM) 


(5.8) 


be Che m>coiiq)onent vector where m«r(M*f 1) ^p and x(k) is the solution to 
equation (4) when x(0) ■ 8. 






I* 
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Ut 


and 


Then 


Let 






1 ( 0 ) 


J[(H) 


v(0) 


v(M) 


i„(S)-6„(S)-I„, 


-^• 


S,«> - 3^ ^<S>A,<S> 


'i 


Then 


<»> ■ MVi4;<s>J%,e>- 


(5.9) 


(5.10) 

(5.11) 

(5.12) 

(5.13) 


(5.14) 


and 


■ 5VTl^;«>J'L^;(S)]-[li(S)]\L•)) 


(5.15) 
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W« note that 


end 




4(§)-^<S)- 


(5.16) 


(5.17) 


Let 




By eqs. (5.S)» (5. II), (5.12), end (5.13), 


\(S) ■ 5^(8) - C4«> -V 


or 


Let us define 


(5.18) 


" M + l4l “ (M ^ \ ”*‘m + 1 ^ ^ (5.19) 


L (§) - lim E[U,(§)]. 

The following Theorem Is a generalisation of Aokl and Yue's Theorem given 
In [ 1] . 

Theorwtt 5.1 

Let the assumptions given In Section 5.1 hold. 

★ 

Then 11m I^(§) "L (Q) for all Jcfl with probability one. Furthermore, 

» 


<■ 


(5.20) 


Proof. By assumption (2), J}m(§) bounded elements for all JcO and Y 

n “ 

has bounded elements. Hence ^(fi) * bounded elements for all 
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icQ. Ut 


• 


«i(S) 

*j(S) 




Then there exists a K<od such that |f^(§)| for all OsO and for all k. 
Thus 

film K^-rK^ (5.21) 

M-co 

Therefore the limit in (5.20) exists. 

Consider the second term on the right hand side of (5.19). Let 




m 


2 2 

Since EV^«0, EVj^ «o for all k, applying Lemma 2.1. a to our 

situation, we have x ,« f (0)V, , b -M, and 

k k’ M 


00 
I 

k-1 k 


2 2 2 
®*k * ^ir ^^^k 2 2*1 

“2 — ^ ^ ® K , ^<n~ < * 

kSa ^2 kSflk^ 
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Thus 

Jl*k" 5T, *‘'® ptoubtuty on. (5.J2) 

Conflder the diird tern on the tight hand tide of (S.19). 

1 T 1 ® 2 

* V V ■ * r V 

M+l “M "M M+1 lA k 

2 2 

Applying Lemna 2.1.b to this situation, we have X|^"V^ , B|x^| -EV^ ■ 

2 

(X <«. Thus 

PiVl kSl^k probability one. (5.23) 


By (5.21), (5.22), and (5.23), the theorem is proved. 

* 2 * 

If 0 is the true parameter, then L (i|g) -19 ■min L (£). The 
following theorem provides sufficient conditions for the existence of a 
sphere S(0^,p) such that 0^ is the unique minixsal point of J*(§) for all 
OcS(§^,p). The form of the theorem was motivated by the work of 
Pereyra [13}. 

We first recall that the khe Euclidean norm on the r" 

space is defined by 

2 ^ 

^ * forallxsR“. 


Theorem 5.2 


Let 11*11^ denotes the Euclidean norm. For brevity, let 

A-C«ii]- —i— i' (§)• 

/H+1 ” 


Note that A A«N (§) which is an mxp matrix. 
M 
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If (1) there is e 0^«n eueh that " 0 end hence 

(2) noneinguler for ell Mcl*^, end 

(3) there is e sphere S(9^,p)cn with redius p>0 centered 
at 9^ and a set of positive nu8d>ers (X«y»m>iI 0 such that 
for all MCI^ 

(1) 2 4 

(ID (Jj tot til 4«S(*|J.D. 

for »ll «tS(S^.p), 

•k 

(iv) Hn <9)|| ap, for all 9sS(9,p)» and 
M 2 

(v) \(p.p+Y)<l» 


Then (a) 9^ is the unique point in S(9^,p) such that 

Mel and is the unique minimal point for lim . f„ (9)f (9) 

M+ 1 -M ^ 

in S(9^,p). I.e., 9^ is the unique point in S(9^»p) such that 


L*(9) - ra^ • L*<9^) 


(b) 9 is the unique minimal point for lim L (9) in S(9 ,p) with 

"0 M-*oo ” 0 

probability one. 


Proof. A result due to Battle in Herget [7] is: 

. * * 


for all 9sS(9^,p), then (9^) ' 52^112 -« 2 l^ 

*11 particular case, -N*(9) +_L_[^(9)]^J^(9) 

andA2^^).N;(9^). 

*AJJ'(Sq)|| 2 ^M.P‘*‘Y all 9eS(9Q,p) by 3(iii) and (iv) . 
Therefore we have 


<*1 "52>*l2 ‘ 

for «n5^,5^«8(gQ.p). 

ncm tuppose there le another point euch that 

4 *«i> 4 <Si> ■«• 

l.e. 

i?Jb57i Wllj*-®- 

We can write 

io-ii-C'^o’^^oXSo-V 

•C‘(So) 

+ c<(s,)-i*iv3<So-V 

We note that 

iiso- Sill, - iKr‘«„)c*;:«i) -<«„) (s, -V -<;«i)]ii, 

‘ «\ ‘«o>ii,K<Si) «i - V"i 

- iC‘«o’yi<<*i>»2 

*x<nP+Y)||8i-S„ll2+^lteJ(8i)||, 

< iis,-siii,+xii*;;(8,)ii 


for oil 
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W« now wane to ahow that 

‘ (arfe «^i'*i>]C4«,>i*"jjlM7i • 


The final inequality followa eaally from Schwarz Inequality aud propertlea 
of a norm, i.e. lx^^| sllxjl^H^II^ «|lA|i2*l|x||^^ 

We now atate a reault given In [14 ]. If B la a pxp matrix, 

B-Cbjj], th« • 

Now look at 




AA^ 




lOS 

■ j5i A 

■ Ji ill JiV*ti*u%j^ 

• hSi ASliW i 
- tA 

< K for all p) and by 3(11). 

Tharafora 

"<«i>«i'‘'CMh *'%ih WHi'- 

^ 2 

Thus 11b ||fl^(S^)D ■0, and wo coneluda Chat 

M ^ • 

ISo*Sill<IISo-ijll 

Which Is a contradiction, and so wa coneluda 8 ■ • . This connlatas the 

•*1 “0 

proof of part a. 

To prove part (b), wa sac that 

1^(S) ** b* (g) with probability one for all 

and L*($) <• 


for all ^tCi by ihaoroB 5.1 


10 $ 


Thua ^ 

with probability ona. 

If than L*(S) >l'*( 9^)( tharafore thara aodata a 6(Stj^ >0 au^ that 

***( 8 ) - *-*< 8 ^,) >« >0 

Slnea 1^(0)** L*(0) with probability ona, thara axlata a 1^(6) *och that 

|i^(8)-L*(S)| <6/2 with probability one 
and 

<6/2 with probability ona for all M>M^(6) 

Thus 

I <Hj(8) • L*<g)) ♦ (Lj|<8q) • ^*<8q)) I <5 wlA probability ona. 

Now look at 

• I - >•*«» +>•*(»- '•'(V* <Si<*o’ ■ I 

» I + 1 (1^(S > . 1*(S)) . a^(s^) - I 

> 0 with probability one for all H>Mq(6). 

Thia conpietas tha proof of part (b). 

The above theorem givea explicit forma for the deaired aphera S(§^»p); 
however, uae of the Euclidean norm waa naadad in tha proof of tha thaorea 
rather than on arbitrary norm. It ia uauelly difficult to compute the 
indicated bounda uaing the Euclidean norm, and ao we preaant in the 
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following theorem a set of conditions in terns of e more general norm. 
Again, let ||*||^ denote the Euclidean norm on the particular real vector 
space under consideration. We will say that ny other norm, ||*||, is sub- 
ordinate to ||*||2 if ||si|iS||sli 2 ^ lat xsR^, and 

define 


||x|| - Max jx |, 

• Ififp 

then 11*11 is subordinate to ||*|| . This fact makes the computation of all 

OD 2 

Che required bounds particularly sisqple if we use interval arithmetic [ll]. 
Theorem 5.3 

Let II denotes the Euclidean norm, and let II* It be any norm which is 
subordinate to ll’H^* 

If (1) there is a O^sO such that f„(9 ) • 0 for all Msl'*' and hence 

”0 "Tl 0 

(2) ^(§q) Is nonsingular for all Msl^, 

(3) there is a sphere S(0 ,p) with radius p >0 centered at G 

0 ”0 

and a set of positive numbers (\,y*m.» 3) such that for all 
+ 

Msl 

(11) for all GsS(Gq,p) 

for all G.S(G^,p) 

(iv) ||N*(G)|| Sit for all GcS<§Q,p) 
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(v) \(Mip-fY)<l 


Then conclusions (s) snd (b) of Theorem 5.2 ere true. 

Proof: As In the proof of Theorem 5.2, we suppose there Is s 0^tS(9^,p), 

ooch that 

“1 “0 

Itm ;r-L-f '^(O )f^(« ) - 0 
M-,« M+1 -M 

Hence , 

Since 11*11 Is a subordinate to 


lim lU-i— f 
/M-f 1 


( Sj )||- o . 


Again we have 


IISg-Sjl<ll«o-5ill + Ml4;;«i)|| tor .UM.I-" 

Now look at 

* sIbHTi 

Henrc 11m ll^^^)||*0 
Therefore we conclude that 
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^Ich l8 a contradiction, and so 0, Part b is the same as in Theorem 

*“l **0 

5.2. This concludes the proof of the theorem. 

Now let S(0^,p) be a sphere such that 0c8(S^,p) and L (0) "L 

implies 9«0 , e.g. as provided in theorems 5.2 and 5.3. Now consider the 
— “0 

GML estimation problem: find ^cS(0^,p) as an estimate of 9^ where ^ 

is constructed by 




M-0,1,2, 


(5.27) 


S(9^,p) denotes the closure of S(9^,p). 

* OB 

To show that {J„} ^ converges to 9. with probability one, we need 
M M»0 ”"0 

the following Lernna. 

Lenma 5.4 [Wald- Kendal l-AstrSm] [Aokl and Yue, 1] 

A 4D A 00 

Let CSjj}jj.Q constructed by (5.2), then [9 ^}^_q converges to 
9 sQ no with probability one, where Q is defined by 


n*-[9:L*(9) -L*(9^)}. 


Theorem 5.5 

Let S(9^,p) be given by Theorems 5.2 or 5.3 and let 
structed by (5.27). Then the CML estimation sequence 


to 9^ with probability one. 

Proof: Since 9 is the \mique 9 in S(9 ,p) such that L (9) «L (§.) > 

0 C 0 

n ns(9^,p) « [9^] is a singU^on. 

Hence the result follows imnedlately from Lemma 5.4. 
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Corollary 5.6 

Let S(§^tP) be the given in Theorem 5.2 or 5.3, then 0^ is locally 
CML identifiable. 


5.3 An Example of Computing the Region of 
Parameter Identifiability 

We now give an example to illustrate the algorithm of Theorem 5.3. 
Consider the system 


u(k) 


• 


ft 4 


* * 


m 

W^(k+1) 

s 

Til ^ 


w^(k) 

+ 

0 

W2(k+ 1) 


CM 

P 

O 




^3 

» s 


L J 


m 4 


ft * 


y(k) - C 1 0 ] 


w^(k) 

W2<k) 


(5.28) 


z(k) » y(k) + v(k) , k- 0,1,2, 
v(k) is N(O.cj^), o^<®. 


The block diagram of this system is depicted in Fig. 3. The deter- 
ministic part of this system is the same as system (4.42) hence by 

reversing the positions of — and — J; — , and by transforming the initial 

z - Til * ■ 'Hz 

conditions by a similarity transformation, the resulting equivalent system 
will generate the same y(k) sequence. 

T 

Let 0=[w^(0), W2(0), Tij, Ti^, T|^] . Recalling from Sec. 4.3, 
i* = [yj(0), W 2 ( 0 ) + (Ti^ - Tipw^CO) , Ti^, T)^] is a point in n which is 
unresolvablc from 9 by the equivalent transformation. Hence the solution 
to the equation f„(9) » 0 is not unique in R^ and only local identifiability 
can be imposed on the parameters of system (5.28). We applied the 


in 


algorithm of Theorem 5.3 to this example wlth^> (O.S» 0.1* 0.3, 0.7* 

T T 

1.0) . We note that 0^ ■ (0.5, *0.1, 0.7, 0.3, 1.0)^ will give exactly 

the same set of y(k) sequence for any input sequence u(k) and henee is 

unresolvable from 0 . However, the distance from 0^ to 0 is 0.4 by 
“0 -0 “1 ' 

using the norm ||*||^* Therefore 0^ and 0^ are each locally Identifiable 
in the spheres S(0^,p) and S(§^,p) respectively where p<0.4. Using an 
input sequence u(k) « 100 sin (kn/4) for k>0,l,— , the following data 
listed in Table 1 are obtained. 

We see that uniqueness is guaranteed in a sphere of radius p»0.33. 

We know the true radius to be p*0.4 in this example. Again, this 

example illustrates that the computed sphere of parameter identif lability 

is of reasonable size in view of the distance from 0. to 9.. Moreover, 

-1 -0 * 

the region size obtained by applying Theorem 5.3 is larger than that 
obtained by applying Theorem 4.4 since condition 3(vi) of Theorem 4.4 is 
more conservative than condition 3(v) of Theorem 5.3. 


Region of parameter Identif lability of system (5.28) 
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6. CONCLUSIONS AND SUGGESTIONS FOR FURTHER RESEARCH 

Concepts of local identifiability and local CML Idantlflabllity 
of parameters which parametrizes the joint probability density function of 
the observation sequence are established. They are applicable to the 
identification problem of control systems where there are stochastic 
disturbances present. The local least-square identifiability and the 
transfer- function identifiability of parameters of deterministic linear 
dynamic systems are also introduced. Sufficient conditions for their 
identifiability are provided. It has been shown that for single-input, 
zero-state linear systems, we are able to find a region containing the 
true parameter as an Interior point such that the true parameters are 
both locally least-square identifiable and identifiable from the transfer 
function in the same region under certain constant rank assumptions on 
the impulse response matrix and the derivatives of the identification 
criterion. 

By modifying Glover and Willems' theorem in [6l, a theorem is given 
to show that if a system is of minimal dimension, the system parameters 
(including the unknown initial state) are locally identifiable if it has 
unique equivalent system locally when the system is restricted to a given 
parametrization. 

A brief survey on the theory of solving nonlinear least-square prob- 
lems, -t-^-norm, and interval arithmetic is given. Employing these techniques, 
a least-square type identification algorithm for finding explicitly the 
regions of parameter identifiability of general linear deterministic 


IIS 



dynamic systems is developed. A numerical example is included to 
illustrate this algorithm. 

By modifying Herget's result [7]» a theorem providing sufficient 
conditions for local CML identifiability of parameters of general dynamic 
systems with Gaussian-white measurement errors is established. A com- 
putation procedure is provided by the theorem for finding the regions of 
parameter identifiability. It has been shown that with probability one, 
the true parameter vector is the unique extremal point of the ma^iimim 
likelihood function parametrized by the unknown parameter vector and the 
constrained maximun likelihood estimation sequence is consistent in the 
region of parameter identifiability. A numerical example is Included to 
illustrate this eexoputation procedure. 

The system parameter identification problem of Gauss-Markov stochastic 
control systems driven by plant Gaussian-white noise and observed with 
Gaussian-white noise is an an area of further endeavor. 

It has been shown the parameter identification problem of linear 
dynamic systems is equivalent to the initial-state observation problem of 
the quadratic- in- the- state bilinear systems. Hence the observability 
theory of quadratic-in-the-state bilinear systems needs to be studied 
more extensively. 

Since the sufficient conditions for parameter identifiability are 
sensitive to the input sequence, further work in the area of optimal 
input synthesis for system identification may prove fruitful. 
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9. APPmDZX 

This Appwidix eoncsint s listing of ths eosjgutsr program ussd to 
conduct ths psrsnstsr Idsntlflestlon sxanpls of ths psrsoMtsrlssd system 
given In Sec. 4.3. Zt hss bssn written In Fortrm Isngusgs using double 
precision. 

If sctusl Input/output mensurcmsnt date ere svsllsble, "GBHEItATE 
INPUT AND OUTPUT SEQUENCES" In this conputer progrsm should be removed. 
Prq>er dimensioning of the mstrlx srrsys should be noted. 
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1 21 I =1 .NP 
121 Jsl.NP 
1 21 Ksl .NP 





Xf 1 )s0«50 00 
XC2) sO«IO 00 



^ i 



you 


O W y 


«J U U 


X6P( I t J*l> =0*0D 00 
OO 123 K-1*NP 

123 XGP( I • J.l) =XGP(I .Jtl>-ffXB(l.J.K)4-XBSTAfl,J,K)>*XXf<) 
XGPI !• J. n=XGPf I • J« 1 >4^XAf I . J> 

128 CCNTINUE 




12C CCNTTNUE 

DO 1 70 I M tNO 
DO ITO J=1«NP 
XFP( I , J| s0«00 00 
DO 170 Nsf.NP 


170 continue 




=2«0*AMAX1 (HLACl I.HLA(2 ),MLAf3 ) « HLA( 4 I • H|.A( 5 II 


SPEC irv A MATPlX IN INTERVAL FORM 



i 

I 




DC 17 Ks 1 * NP 

CALL INTAODIBI I« J,K) .BST AR ( I , J, K > • Cl > 
17 CCPI I. J.K)=C1 


6 I 




ET«T(| )aCMPLX(Rl «l»l I 
ETAT<2»=CMPt.K< R2 ,R2» 
AX (1 > = ET AE(1 ) 

AX(2 ) =ETAE(2 } 

AX( 3)-FE(l .1 ) 



CALL lNTAD0<C3*GAMMAft.J,2> .C2) 
GAMMA( I . J,2 I =C2 
CONTINUE 
DC 2 1 I=ItNP 



GAMMPf I , J,K,1 ) =c »*PLX ( 0 * 0 * 0*0 I 

21 GAMMPf I • j,k« 2) sGOP( I • J,K) 
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